Method and system for classifying and processing of pixels of image data

ABSTRACT

A system and method classify a pixel of image data as one of a plurality of image types. A first image characteristic value for the pixel, a second image characteristic value for the pixel, a third image characteristic value for the pixel, and a fourth image characteristic for the pixel is determined. Some of these determinations may be resolution dependent. The values from these determination are utilized in assigning an image type classification to the pixel. Moreover, if at least one of the image characteristic values is greater than a predetermined threshold value the pixel is classified as a halftone peak value. The system includes a plurality of microclassifiers for determining a distinct image characteristic value of the pixel; a plurality of macroreduction circuits connected to the plurality of microclassifiers for performing further higher level operations upon the distinct image characteristic values of the pixel to produce reduced values; and a classification circuit to classify the pixel as an image type based on the reduced values from the macroreduction circuits. The system also includes a circuit to detect flat peaks without detecting multiple peaks and a rectangular blur filtering system.

FIELD OF THE PRESENT INVENTION

The present invention relates generally to a system for processingdocument images, and more particularly, to an improved method of imageprocessing the document images utilizing a fuzzy logic classificationprocess.

BACKGROUND OF THE PRESENT INVENTION

In the reproduction of images from an original document or images fromvideo image data, and more particularly, to the rendering of image datarepresenting an original document that has been electronically scanned,one is faced with limited reflectance domain resolution capabilitiesbecause most output devices are binary or require compression to binaryfor storage efficiency. This is particularly evident when attempting toreproduce halftones, lines, and continuous tone (contone) images.

An image data processing system may be tailored so as to offset thelimited reflectance domain resolution capabilities of the renderingapparatus, but this tailoring is difficult due to the divergentprocessing needs required by different types of images which may beencountered by the rendering device. In this respect, it should beunderstood that the image content of the original document may consistof multiple image types, including halftones of various frequencies,continuous tones (contones), line copy, error diffused images, etc. or acombination of any of the above, and some unknown degree of some or allof the above or additional image types.

In view of the situation, optimizing the image processing system for oneimage type in an effort to offset the limitations in the resolution andthe depth capability of the rendering apparatus may not be possible,requiring a compromised choice which may not produce acceptable results.Thus, for example, where one optimizes the system for low frequencyhalftones, it is often at the expense of degraded rendering of highfrequency halftones, or of line copy, and visa versa.

To address this particular situation, “prior art” devices have utilizedautomatic image segmentation to serve as a tool to identify differentimage types or imagery. For example, in one such system, imagesegmentation was addressed by applying a function to the video, theoutput of which was used to instruct the image processing system as tothe type of image data present so that it could be processedappropriately. In particular, an auto-correlation function was appliedto the stream of pixel data to detect the existence and estimate thefrequency of halftone image data. Such a method automatically processesa stream of image pixels representing unknown combinations of high andlow frequency halftones, contones, and/or lines. The auto-correlationfunction was applied to the stream of image pixels, and for the portionsof the stream that contain high frequency halftone image data, thefunction produced a large number of closely spaced peaks in theresultant signal.

In another auto-segmentation process, an auto-correlation function iscalculated for the stream of halftone image data at selected time delayswhich are predicted to be indicative of the image frequencycharacteristics, without prior thresholding. Valleys in the resultingauto-correlated function are detected to determine whether a highfrequency halftone image is present.

An example of a “prior art” automatic segmentation circuit isillustrated in FIG. 6. The basic system as shown in FIG. 6 is made up ofthree modules. Input information stored in a data buffer 10 issimultaneously directed to an image property classifying section 20, thefirst module, and an image processing section 30, the second module. Theimage property classifying section 20, is made up of any number ofsubmodules, (e.g. auto-correlator 21 and discriminator 22), whichdetermine whether a block of image pixels stored in the data buffer 10is one type of imagery or another, (e.g. halftone, line/text, orcontone). In parallel with the image property classifying section 20,the image processing section 30 is made up of any number ofsub-processing sections, (e.g. high frequency halftone processor 31 lowfrequency halftone processor 32, line/text processor 33, or contoneprocessor 34), which perform image processing operations on the sameblock of image pixels as section 20. Each image sub-processing sectionperforms image processing operations that are adapted to improve theimage quality of a distinct class of imagery. The third module, controlsection 41, uses the information derived from the image classifyingsection 20, to control the image processing section 30. In other words,the control section 41 acts like a multiplexer and selects the properprocessed image data according to the image classification determined bythe image classifying section 20.

The decision as to what class of imagery image data belongs to istypically binary in nature. For example, in a conventional imagesegmentation scheme image property classifying section 20 classifiesimage data as one of three classes of imagery, (high frequency halftone,low frequency halftone, or contone). Depending on those classification,image data is processed according to the properties of that class ofimagery is selected, (either low pass filter and re-screening if it's ahigh frequency halftone, threshold with a random threshold if it is alow frequency halftone, etc.). Also, assuming that the decision as towhich of the three classes of imagery image data belongs is based on asingle image property, the peak count of the input image data, theresulting image classification decision of the peak count image propertyis made by thresholding the peak count into three classes of imagery.

Consequently, the control section 40 decides the type of imageprocessing the image data requires depending on the decision made by theclassification section 20. Thus, the output of classification section 20is quantized to one of three possibilities. The control section 40selects the output from one of the three image sub-processing sectionsbased upon this classification.

Based on the nature of conventional image classification systems, theclassifying information, gathered over a context of many pixels, changesgradually. But in the process of comparing this classifying informationwith a classification threshold one could create abrupt change in theclasses. This abrupt decision making, which produces a forced choiceamong several distinct alternative choices, is a primary reason for theformation of visible artifacts in the resulting output image. Mosttransition points or thresholds are selected so that an image can beclassified as one class of imagery with a high degree of certainty;however, those classes of imagery that cannot be classified with suchcertainty have multiple transition points or a transition zone.

Using only one point to define a transition zone results in theformation of visible artifacts in the resulting output image if theoutput image spans in the transition zone. Although it is possible toshift or make the transition zone narrower so that there is less chancethat an image falls into the zone, there exists limitations on hownarrow the zone can be made. The narrowing of the transition zone is thedecreasing of noise and/or variation in the information used to classifyso as to narrow the area over which classification is not “certain”,resulting in less switching between classifications.

Moreover, the classification of real images covers a continuum from wellbelow to well above thresholds between classifications. This means thatthere are areas of an image which are, for example, just above athreshold. Variations in the gathered (lowpass filtered) information dueto “flaws” in the input video or ripple due to interactions between thearea of image being used for the classification process and periodicstructures in the input video results in areas falling below thethreshold. With discrete classification, this results in a drasticallydifferent classification, thereby resulting in artifacts in the renderedimage.

Thus, it is desirable to classify image data in a fuzzy manner, slowlysliding the classification from one classification to the other,reflecting the information that has been gathered. Artifacts in theresulting rendered image will now be soft and follow the contours of theimage, and so the artifacts will not be objectionable.

In general, the “prior art” describes the control section 40 asessentially having a switch. Since the image processing steps performedfor each class of imagery are different depending on the classificationgiven to each block of input image pixels, the switch or multiplexerallows data residing at the output of the image processor 30 to bedirected to an output buffer 50 depending on the decisions made by theimagery classifying section 20 which are received as signals on lines 23and 24. This type of binary decision making is rigid and results inimage segmentation decisions that do not fail gracefully andconsequently form visible artifacts in the output image.

To address this forming of visible artifacts in the rendered outputimage, it has been proposed to utilize a probabilistic segmentationprocess to allow the image processing system to fail more gracefullywhen incorrect segmentation decisions are made. An example of such aprobabilistic segmentation system is illustrated in FIG. 2.

FIG. 2 shows a block diagram of a conventional image processing systemwhich incorporates a probabilistic classification system. As illustratedin FIG. 2, the conventional system receives input image data derivedfrom any number of sources, including a raster input scanner, a graphicsworkstation, an electronic memory, or other storage elements, etc. Ingeneral, the image processing system shown in FIG. 2 includesprobabilistic classifier 25, image processing section 30, an imageprocessing and control mixer 41.

Input image data is made available to the image processing system alongdata bus 15, which is sequentially processed in parallel byprobabilistic classifier 25 and image processing section 30.Probabilistic classifier 25 classifies the image data as a ratio of anumber of predetermined classes of imagery. The ratio is defined by aset of probability values that predict the likelihood the image data ismade up of a predetermined number of classes of imagery. Theprobabilities 27, one for each predetermined class of imagery, are inputto the image processing mixer or control unit 41 along with image outputdata from image processing section 30.

Image processing section 30 includes units 31, 32, and 34 that generateoutput data from the image data in accordance with methods unique toeach predetermined class of imagery. Subsequently, mixer 41 combines apercentage of each class of output image data from units 31, 32, and 34according to the ratio of the probabilities 27 determined by classifier25. The resulting output image data for mixer 41 is stored in outputbuffer 50 before subsequent transmission to an image output terminalsuch as a printer or display.

Initially, the stream of image pixels from an image input terminal (IIT)is fed to data buffer 10. The image data stored in buffer 10 is in rawgrey format, for example, 6 to 8 bits per pixel. A suitable block sizeis 16 pixels at 400 spots per inch, or 12 pixels at 300 spots per inch.Too large of a sample size results in the inability to properly switchclassification in narrow channels between fine structures in the image,or to switch soon enough when moving from one classification to another.An example of this problem is small text forming a title for a halftoneimage. Given a font size which is large enough to read, a good layoutpractice of leaving white space which is at least a half a line betweenthe text and the image, a one millimeter block turns out to be a goodcompromise with most documents. Thus, too large a sample size results inclassification transitions at the edge of objects to be larger than thewhitespace between the objects, resulting in inappropriateclassification and rendering.

With reference FIG. 3, the conventional probabilistic classifier 25 isshown in detail. The block of image pixels stored in buffer 10 istransmitted to a characteristic calculator 28 through data buffers 15.Calculator 28 provides an output value that characterizes a property ofthe image data transmitted from buffer 10, such as its peak count. Inone embodiment, a characteristic value is determined by calculator 28that represents the peak count of the block of image data. The peakcount is determined by counting those pixels whose values are thenon-trivial local area maximum or minimum in the block of image data.First local area maximum or minimum pixel values are selected dependingon whether the average value of all the pixels in the block of imagedata is lower or higher than the median value of the number of levels ofeach pixel.

After calculator 28 evaluates the peak count of the image data,probability classifier 29 determines three probability values 27 thatcorrespond to each image type associated with the peak count asexpressed by the characteristic function stored in memory 26. Thecharacteristic function, determined with apriori image data, representsa plurality of probability distributions that are determined using apopulation of images. Each probability distribution depicts theprobability that a block of image data is a certain type given theoccurrence of an image property, a peak count.

For example, the characteristic function stored in memory 26 can berepresented by the graph shown in FIG. 4, which relates the probabilitydistributions for a contone 1, low frequency halftone 2, and highfrequency halftone 3 to the occurrence of a particular imagecharacteristic, which in this example is a peak count. Thecharacteristic function stored in memory 26 can be adjusted using inputcontrol 18. Using control 18, the resulting output image stored inbuffer 50 can be altered by modifying the characteristic functionrepresenting the different classes of imagery evaluated by the imageprocessing system 30.

Subsequently, probability classifier 29 determines each probabilityvalue by evaluating the probability distribution of each image typerepresented by the characteristic function stored in memory 26. Afterdetermining the probability values, classifier 29 outputs these resultsto image processing mixer or control 41.

The image processing section of FIG. 2 operates concurrently with theprobabilistic classifier 25 on the image data stored in buffer 10. Imageprocessing section 30 includes a high frequency halftone processing unit31, a low frequency halftone processing unit 32, and a contoneprocessing unit 34. Each processing unit processes all image data inaccordance with a particular image type. Each of the processing units31, 32, and 34 generates output blocks of unquantized video data.

Image processing control 41 mixes the data output blocks to form acomposite block of output image signals that is stored in output buffer50. The manner in which the output blocks are mixed is characterized bya ratio defined by the probability determined by the probabilisticclassifier 25.

FIG. 5 shows the conventional image processing mixer 41 in detail. Mixer41 multiplies the output blocks with the probability, using multipliers42, 43, 44. The resulting output from each multiplier is representativeof a percentage or ratio of each output block, the sum of which definesa composite block of output image signals. The composite block of outputimage signals is formed by adding the output of the multipliers usingadder 45 and by subsequently quantizing the sum of adder 45 usingquantizer 47. The resulting image block output by quantizer 47 is storedin output buffer 50 before subsequent transmission for output to animage output terminal having limited resolution or depth.

The above-described image classification system utilizes a probabilisticapproach to classify the image data. Such an approach presents problemsin that the classification of the image data is mutually exclusive, theimage data is classified as a particular type in absolute termseventhough the probability of the decision being correct is just over50%. This results in difficulties in trying to design an imageprocessing system which will process the image data without visibleartifacts in the rendered image when the decision on the image type doesnot have a high confidence.

Not only is image classification important to a digital reprographicsystem, rendering based on this classification is important. One suchcomponent of the rendering system is digital filtering. The digitalfiltering process should be both efficient and low cost. Moreover, thefilter design should have some non-separable and/or time-varyingcharacteristics so that the filter can be used in a fuzzy segmentationsystem. However, trying to achieve one goal or another can adverselyimpact the other goal. Various approaches have been devised for theimplementation of digital filtering techniques which try to solveminimize the adverse impacts. These techniques will be discussed brieflybelow.

In a “prior art” digital filtering technique, a two-dimensional finiteimpulse response filter having a plurality of filter portions ofessentially identical construction are arranged in a parallelconfiguration. A de-multiplexer separates an input data signalcomprising consecutive digital words and supplies each digital word insequence to a separate filter portion. Subsequently, a multiplexer,coupled to the output of the filter portions, selectively outputs thefiltered data from each filter portion in a sequence corresponding tothe order of separation of the input data, thereby resulting in afiltered version of the original input data.

The system described above all has the limitation with respect to eitherspeed or high cost. In view of these limitations, it has been proposedto provide a plurality of one-dimensional transform units that may beselectively combined with an additional one-dimensional transform unitto produce a plurality of distinct two-dimensional filters, any one ofwhich is selectable on a pixel by pixel basis. Moreover, this proposedconventional system has the added advantage of providing two-dimensionalfinite impulse response filters without employing multiple, identicallyconstructed two-dimensional filters arranged in a parallel fashion,thereby substantially reducing the complexity and cost of the filterhardware. To get a better understanding of this conventional system, theconventional system will be described below.

The conventional system, as illustrated in FIG. 1, includes imageprocessing module 20 which generally receives offset and gain correctedvideo through input line 22. Subsequently, the image processing module20 processes the input video data according to control signals from CPU24 to produce the output video signals on line 26. As illustrated inFIG. 1, the image processing module 20 may include an optionalsegmentation block 30 which has an associated line buffer 32,two-dimensional filters 34, and an one-dimensional rendering block 36.Also included in image processing module 20 is line buffer memory 38 forstoring the context of incoming scanlines.

Segmentation block 30, in conjunction with the associated scanlinebuffer 32, automatically determines those areas of the image which arerepresentative of halftone input region. Output from the segmentationblock, (video class), is used to implement subsequent image processingeffects in accordance with a type or class of video signals identifiedby the segmentation block. For example, the segmentation block mayidentify a region containing data representative of an input highfrequency halftone image, in which case a lowpass filter would be usedto remove screen patterns, otherwise, a remaining text portion of theinput video image may be processed with an edge enhancement filter toimprove fine line and character reproduction when thresholded.

Two-dimensional filter block 34 is intended to process the incoming,corrected video in accordance with the predetermined filteringselection. Prior to establishment of the required scanline content, theinput video bypasses the filter by using a bypass channel within thetwo-dimensional filter hardware. This bypass is necessary to avoiddelirious effects to the video stream that may result from filtering ofthe input video prior to establishing the proper context.

Subsequent to two-dimensional filtering, the one-dimensional renderingblock is used to alter the filtered, or possibly unfiltered, video datain accordance with selected one-dimensional video effects.One-dimensional video effects include, for example, thresholding,screening, inversion, tonal reproduction curve (TRC), pixel masking,one-dimensional scaling, and other effects which may be appliedone-dimensionally to the steam of video signals. As in thetwo-dimensional filter, the one-dimensional rendering blocks alsoincludes a bypass channel where no additional effects would be appliedto the video, thereby enabling the received video to be passed throughas an output video.

Therefore, it is desirable to implement an image classification systemwhich provides a truer classification of the image type and the imagetypes are not necessarily mutually exclusive. Such a system wouldincorporate fuzzy logic, thereby allowing image data to be classified asbeing a member of more than one image class. This feature is critical inareas where the image goes from one image type to another. Moreover, itis desirable to implement a image processing and rendering system whichtakes advantage of the fuzzy classification system.

SUMMARY OF THE PRESENT INVENTION

One aspect of the present invention is a system for classifying a pixelof image data as one of a plurality of image types. The system includesa plurality of microclassifiers, each microclassifier determining adistinct image characteristic value of the pixel; a plurality ofmacroreduction circuits operatively connected to the plurality ofmicroclassifiers, each macroreduction circuit performing further higherlevel operations upon the distinct image characteristic values of thepixel to produce reduced values; and a classification circuit toclassify the pixel as an image type based on the reduced values from themacroreduction circuits.

A second aspect of the present invention is a system for classifying apixel of image data as one of a plurality of image types. The systemincludes first means for determining a plurality of distinct imagecharacteristic values of the pixel; second means, operatively connectedto the first means, for performing further higher level operations uponthe distinct image characteristic values of the pixel to produce reducedvalues; and a classification circuit to classify the pixel as an imagetype based on the reduced values from the macroreduction circuits.

A third aspect of the present invention is a system for determining amembership value for a classification vector associated with a pixel ofimage data. The system includes a plurality of microclassifiers, eachmicroclassifier determining a distinct image characteristic value of thepixel; a plurality of macroreduction circuits operatively connected tothe plurality of microclassifiers, each macroreduction circuitperforming further higher level operations upon the distinct imagecharacteristic values of the pixel to produce reduced values; and aclassification circuit to assign a membership value to theclassification vector associated with the pixel based on the reducedvalues from the macroreduction circuits.

A fourth aspect of the present invention is a printing system forrendering a pixel of image data according to one of a plurality of imagetypes. The system includes a plurality of microclassifiers, eachmicroclassifier determining a distinct image characteristic value of thepixel; a plurality of macroreduction circuits operatively connected tothe plurality of microclassifiers, each macroreduction circuitperforming further higher level operations upon the distinct imagecharacteristic values of the pixel to produce reduced values; aclassification circuit to classify the pixel as an image type based onthe reduced values from the macroreduction circuits; processing meansfor image processing the pixel according to the image typeclassification; and print means for rendering the processed pixel.

Further objects and advantages of the present invention will becomeapparent from the following descriptions of the various features of thepresent invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The following is a brief description of each drawing used in describingthe present invention, thus, the drawings are being presented forillustrative purposes only and should not be limitative of the scope ofthe present invention, wherein:

FIG. 1 is a schematic illustration of a conventional image processinghardware module incorporating a two-dimensional filter;

FIG. 2 is a block diagram illustrating a conventional image processingsystem incorporating probabilistic segmentation;

FIG. 3 is a block diagram detailing the probabilistic segmentor shown inFIG. 2;

FIG. 4 shows an example of a characteristic function of the imageproperty, peak count;

FIG. 5 is a block diagram illustrating in detail the image processingmixer shown in FIG. 2;

FIG. 6 is a block diagram illustrating a “prior art” image processingsystem;

FIG. 7 is a block diagram of a digital filtering system incorporatingthe fuzzy classification process of the present invention;

FIG. 8 is a block diagram illustrating a screening system incorporatingthe fuzzy classification process of the present invention;

FIG. 9 is a block diagram illustrating a more detailed version of thesystem illustrated in FIG. 8;

FIG. 10 is a block diagram illustrating a scalable image classificationsystem architecture according to the concepts of the present invention;

FIG. 11 is a block diagram illustrating another scalable imageclassification system architecture according to the concepts of thepresent invention;

FIG. 12 is a block diagram illustrating a third scalable imageclassification system architecture according to the concepts of thepresent invention;

FIG. 13 is a graphical representation of an enlarged view of a halftonearea;

FIG. 14 is a graphical representation of detected peaks for thegraphical representation of FIG. 13;

FIG. 15 is a graphical representation of an enlarged view of a ladderchart;

FIG. 16 is a graphical representation of false detection of halftonepeaks for the graphical representation of FIG. 15;

FIG. 17 is a graphical representation of a reduction in the falsedetection of halftone peaks for the graphical representation of FIG. 15according to the concepts of the present invention;

FIG. 18 is a graphical representation of an enlarged view of a kanjiarea;

FIG. 19 is a graphical representation of false detection of halftonepeaks for the graphical representation of FIG. 18;

FIG. 20 is a graphical representation of a reduction in the falsedetection of halftone peaks for the graphical representation of FIG. 18according to the concepts of the present invention;

FIG. 21 is a graphical representation of a comparison between afrequency response of a triangular filter and a rectangular filteraccording to the concepts of the present invention;

FIG. 22 is a block diagram illustrating an architecture for a paralleltwo-dimensional blur filter according to the concepts of the presentinvention;

FIG. 23 is a circuit diagram of a filter circuit according to theconcepts of the present invention; and

FIG. 24 is a block diagram illustrating an architecture for atwo-dimensional blur filter.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

The following will be a detailed description of the drawings illustratedin the present invention. In this description, the terms “image data” or“pixels” in the form of video image signals, which may be either analogor digital voltage representations of an image, indicate arepresentation of an image provided from a suitable source. For example,the image signals may be obtained through line by line scanning of animage bearing the original by one or more photosensitive elements, suchas a multiple photosite array of charge couple devices commonly referredto as CCDs. Line by line scanning of an image bearing the original forthe duration of image data is well known and does not form a part of thepresent invention.

Image data may also be derived by a computer workstation program inaccordance with document creation application software or from a datastorage device. In content, the original video image signals may becomposed entirely of a single image component such as lines, text, lowfrequency halftones, high frequency halftones, contones, or anycombination thereof.

The following description also includes references to slowscan andfastscan digital image data when discussing the directionality oftwo-dimensional filtering architecture. For purposes of clarification,fastscan data is intended to refer to individual pixels located in asuccession along a raster of image information, while slowscan datarefers to data derived from a common raster position across multiplerasters or scanlines.

As an example, slowscan data would be used to describe signals capturedfrom a plurality of elements along a linear photosensitive array asarray is moved relative to a document. On the other hand, fastscan datawould refer to the sequential signals collected along the length of thelinear photosensitive array during a single exposure period which isalso commonly referred to as a raster of data.

Moreover, in describing the present invention, it is assumed that thevideo signal has a value in a range between 0 and 255. However, anyrange from the video signal can be utilized in conjunction with thepresent invention. Furthermore, in the following description, the term“grey level” will be used to describe both black and white and colorapplications.

Furthermore, in describing the present invention, the term “pixel” willbe utilized. This term may refer to an electrical, (or optical, if fiberoptics are used), signal which represents the physical measurableoptical properties at a physical definable area on a receiving medium.The receiving medium can be any tangible document, photoreceptor, ormarking material transfer medium.

Moreover, the term “pixel” may refer to an electrical, (or optical, iffiber optics are used), signal which represents the physicallymeasurable optical properties at a physically definable area on thedisplay medium. A plurality of the physically definable areas for bothsituations represent the physically measurable optical properties of anentire physical image to be rendered by either a material markingdevice, electrically or magnetic marking device, or optical displaydevice.

Lastly, the term “pixel,” in the scanning environment, may refer to anelectrical, (or optical, if fiber optics are used), signal whichrepresents physical optical property data generated from a signalphotosensor cell when scanning a physical image so as to convert thephysical optical properties of the physical image to an electronic orelectrical representation. In other words, in this situation, a pixel isan electrical, (or optical), representation of the physical opticalproperties of a physical image measured at a physical definable area ona optical sensor. The term “pixel,” in the rendering environment, mayrefer to an electrical, (or optical, if fiber optics are used), signalwhich represents the smallest physical area on a recording substratethat can be rendered. In other words, in this situation, a pixel is anelectrical, (or optical), representation of the physical spot created bya laser in a laser printed or the physical spot created by the smallestrendered ink droplet.

Many of the documents produced today are compound documents in that thedocuments are composed of several different sub-images that are ofdifferent image types or image classes. Some of the common types aretext, photos (contones), and halftones. One reason for the increasedappearance of compound documents is the widespread use of commerciallyavailable word processing and desktop publishing software that is ableto generate them.

As is well known, different types of images require different processingin order to provide optimal image quality. Conventionally, toautomatically choose the best processing for different areas of animage, each area is classified into one of several pre-defined classesto determine how to render that part of the image. This image type orimage class information can then be used to determine the appropriateprocessing required to obtain a good rendition of the image whenprinting, to choose a method of image compression, to determine ifoptical character recognition would be useful, etc.

However, as noted previously, the classification process should not beso crisp so as to avoid problems when the input image is not verysimilar to any of the classes, or the input images properties straddlethe border between two classes.

For example, if a particular action is taken based upon a single classidentification because the classes are mutually exclusive, it may createundesirable results for a non-prototype image. This is seen whenrendering images for printing on a xerographic printer. Theclassification of the image can cause output artifacts such as when ahalftone image is classified as a contone image.

Another type of problem is that adjacent areas of the image may beclassified differently due to small variations in the image. This iscalled class switching. If this information is used for imageenhancement and printing, the output may have objectionable artifactsdue to local variations. Examples of these objectionable artifacts aregrainy image outputs.

To eliminate the above described problems, an image classificationsystem which utilizes a fuzzy membership into each category or class canbe used. In other words, the classes in a fuzzy classification systemare not mutually exclusive, thereby eliminating problems with classswitching and also allowing those areas to have processing differentthan that of any of the other pre-defined classes; i.e., the output canchoose between a continuum of possible image processing techniques.

In standard classification techniques, each area has one class assignedto it. In the fuzzy implementation of the present invention, each areahas a classification vector assigned to it. Every element of theclassification vector has a membership value associated with each of thepre-defined prototype classes.

Similar to the creation of crisp classifiers, a set of heuristic rulesare used to determine the form of the classifier. The following is anexample of how heuristic rules are used to create a fuzzy classifier,according to the concepts of the present invention.

For illustrative purposes, an example of a two class non-fuzzy system isdiscussed. In this example, the system only classifies a particularregion as either contone (i.e., grey pictorial) or text. An image may beconsidered text if there are a lot of edges and most pixels are black orwhite. If this is not true, the picture is considered contone.

In order to determine edges, a variable relating to the Laplacian of theimage data at every point (pixel) is used. A typical implementation forthis type of segmentation may be if the summation of the squares of theLaplacian at every pixel in the subblock is greater than a predeterminedsummation threshold and the sum of the percentage of pixels with greyvalue less than a black threshold value and the percentage of pixelswith grey value greater than a white threshold value is greater than apredetermined bi-modal threshold, the image is text, else the image iscontone.

In this example, since the parameters are device dependent, tests, whichare known to those skilled in the art, would be run to determine thevalues of all of the parameters; the percentage of pixels with greyvalue less than a black threshold value, the percentage of pixels withgrey value greater than a white threshold value, the summationthreshold, and the bi-modal threshold before executing the segmentationroutine. Note that only one class can be chosen, either text or contone.

In order to implement a fuzzy classifier, according to the concepts ofthe present invention, several modifications must be made to the abovedescribed heuristic rules. As described above, there exists a singlerule defining only text. If the condition for text is “not true,”contone is chosen. In the fuzzy system of the present invention, contonemust have it's own rule, since the text membership rule is not anabsolute truth, but a relative truth that the classification is true.

Moreover, even providing a rule for contones will not satisfy theexcluded middle law; therefore, a third “other” class must be added tosatisfy the constraints of fuzzy logic. Without the “other” class, itwould be possible to have membership of the image in all classes be verysmall. Thus, the “other” class creates a lower bound of a half (0.5) forthe minimum membership in any given class. A minimum magnitude for themaximum membership assures that all actions/decisions made using therelative membership values are not extremely sensitive to the classmemberships, which they would be if membership in all classes was small,thereby making the fuzzy classification more robust.

In the fuzzy classification scheme of the present invention, the pixelor unit of image data has a membership in each of the three classes;text, image and “other.” In other words, the pixel is no longerconsidered to be an element of just one of the mutually exclusiveclasses. However, if the determination for one class reach absolutecertainty; i.e. the membership in a single class is 1 and the otherclasses is zero; the fuzzy system does generate values which wouldrepresent a crisp system.

In view of this non-exclusivity characteristic of the pixel imagemembership, the membership of the pixel is represented by a membershipvector, V_(i), whose entries correspond to the membership of the pixel(image element) in each of the classes. Note, typically, there are noconstraints on this vector other than all of it's elements must begreater than or equal to 0 and less than or equal to 1. However, sincethe fuzzy classification rules of the present invention have been setupwith a third “other” class, at least one of the elements of the vectormust be greater than or equal to 0.5 and less than or equal to 1.

Using the two class example above, the fuzzy classification rules wouldbe the following. If the summation of the squares of the Laplacian atevery pixel in the subblock is greater than a predetermined summationthreshold and the sum of the percentage of pixels with grey value lessthan a black threshold value and the percentage of pixels with greyvalue greater than a white threshold value is greater than apredetermined bi-modal threshold, the pixel would be assigned amembership value for the “text” class which is the minimal valueassociated with each of the conditional statements.

To better understand this concept, the following brief explanation offuzzy logic will be provided. In fuzzy logic, unlike Boolean logic, theresults of the conditional statements do not generate either absolutetrue or absolute false, but a value corresponding to the amount of theresulting statement which is true. This result is due to the fact thatthe conditional statements are also not absolute.

For example, in the above described rule, from testing, it may bedetermined that the midpoint (predetermined target condition value) ofthe fuzzy Laplacian summation condition should be 50. The midpointrepresents maximum uncertainty as to whether the value 50, in thisexample, is a member of the class Large Laplacian. Moreover, fromtesting, it is determined that, with absolute certainty, a pixel ismember of the Large Laplacian (membership equals 1.0) if the summationis equal to or greater than 75 (predetermined absolute condition value)and it is determined that, with absolute certainty, a pixel is not amember of the Large Laplacian (membership equals 0.0) if the summationis equal to or less than 25 (predetermined absolute condition value).Fuzzy logic allows the classifier to assign 0.5 (conditional value) to aresult where the summation is 50 and linearly extrapolate to theassigned values (conditional values) to 1 and 0.0 for the values 75 and25, respectively; i.e., value 55 would be assigned a membership value of0.6. Note that these values are device dependent, and thus, the midpointand the range needs to be determined for each individual device.

Furthermore, from testing, it may be determined that the midpoint of theclass bi-modal should be 80; with absolute certainty, a pixel is in themembership if the percentage sum is equal to or greater than 90; and,with absolute certainty, a pixel is not in the membership if thepercentage sum is equal to or less than 70. Fuzzy logic allows theclassifier to assign 0.5 to a result where the sum value is 80 andlinearly extrapolate to the assigned values to 1 and 0.0 for the values90 and 70, respectively; i.e., value 85 would be assigned a membershipvalue of 0.75. Note that these values are device dependent, and thus,the midpoint and the range needs to be determined for each individualdevice.

To further explain the fuzzy technique, it is assumed that themembership values for each conditional statement are 0.5 and 0.33,respectively. In this scenario, the membership value for the pixel forthe class text would be 0.33 because fuzzy logic treats “ANDed”statements as determining the minimal value for all the conditions andassigning the minimal value to the membership value.

Using the contone rule of if the summation of the squares of theLaplacian at every pixel in the subblock is less than a predeterminedsummation threshold and the sum of the percentage of pixels with greyvalue less than a black threshold value and the percentage of pixelswith grey value greater than a white threshold value is less than apredetermined bi-modal threshold, the image is “contone,” eachconditional statement will be discussed in fuzzy logic terms.

For example, in the above described rule, from testing, it may bedetermined that the midpoint of the fuzzy Laplacian summation conditionshould be 50. Moreover, from testing, it is determined that, withabsolute certainty, a pixel is in the membership if the summation isequal to or less than 25 and it is determined that, with absolutecertainty, a pixel is not in the membership if the summation is equal toor greater than 75. Fuzzy logic allows the classifier to assign 0.5 to aresult where the summation is 50 and linearly extrapolate to theassigned values to 1 and 0.0 for the values 25 and 75, respectively;i.e., value 55 would be assigned a membership value of 0.4.

Furthermore, from testing, it may be determined that the midpoint of thefuzzy bi-modal condition should be 80; with absolute certainty, a pixelis in the membership if the sum is equal to or less than 70; and, withabsolute certainty, a pixel is not in the membership if the sum is equalto greater 90. Fuzzy logic allows the classifier to assign 0.5 to aresult where the percentage value is 80 and linearly extrapolate to theassigned values to 1 and 0.0 for the values 70 and 90, respectively;i.e., value 85 would be assigned a membership value of 0.25.

To further explain the fuzzy technique, it is assumed that themembership values for each conditional statement are 0.75 and 0.8,respectively. In this scenario, the membership value for the pixel forthe class text would be 0.75 because fuzzy logic treats “ANDed”statements as determining the minimal value for all the conditions andassigning the minimal value to the membership value.

Lastly, the fuzzy rules states that if image is neither “text” or“contone,” the image is “other.” This last rule, which defines the“other” class, can be represented mathematically asμ_(other)(image)=min(1-μ_(text)(image), 1-μ_(contone)(image)) whereμ_(x)(Y) is the membership of Y in the class X. Note that ifμ_(text)(image), and μ_(contone)(image) are smaller than 0.5 thenμ_(other)(image) will be greater than 0.5 (as stated earlier). In theexample, given above, μ_(text)(image) is equal to 0.33 andμ_(contone)(image) is equal to 0.75, thus, μ_(other)(image) would beequal to 0.25, with the resulting membership vector being [0.33 0.750.25]. Note the element values of the vector need not add upto 1.

The predicate of each the rules described above is extended to a fuzzytruth instead of an absolute truth to provide the element value for themembership vector. Thus, in order to make the inequality “Y is >X” afuzzy truth, a membership function is defined for “>X”. Similarly, afuzzy membership rule can be defined for <X (very often, the membershipin (<X) is equal to not (>X): (1—membership of (>X)).

For simplicity in implementation, the membership in (>X) is defined asfollows: $\begin{matrix}{{\mu_{> X}(Y)} = \quad \left( {1,} \right.} & {\quad {{Y > {X + {\Delta \quad X}}},}} \\{\quad {0,}} & {\quad {{{Y <}\quad = {X - {\Delta \quad X}}},}} \\{\quad {{\left( {Y - X + {\Delta \quad X}} \right)/\left( {2\quad \Delta \quad X} \right)},}} & \left. \quad {{{X - {\Delta \quad X}} < Y <}\quad = {X + {\Delta \quad X}}} \right)\end{matrix}$

The value of ΔX determines the level of fuzzification of the class; ifΔX is extremely small, then the definition reduces to the crispdefinition of greater than. It is further noted that although thefuzzification has been described as a linear relationship, the functiondescribing the values between the end points and the mid point may beany type of function. Moreover, the midpoint could represent absolutecertainty in the class and have a membership value of 1 and theendpoints represent absolute certainty of non-membership such that themembership values would graphically form a triangle with the midpointbeing the peak.

Returning to the multiple “If's” in the above rules, the membership ofimage in the class text is equal to the fuzzy value of the predicate,μ_(text)(image)=min(μ_(>Slp Threshold)(Slp²),μ_(>Bimodal Threshold)(White+Black)).

To expand the concepts of the present invention to the processing ofimages on a typical xerographic laser printer requires separating imagesinto several classes; for example, white, black, edge, pictorial, lowfrequency halftone, mid frequency halftone, high frequency halftone andother, etc. The classes white, black, and pictorial are subclasses ofthe set “contone” and low frequency halftone, mid frequency halftone,high frequency halftone are subclasses of the set “halftone.”

In a preferred embodiment of the present invention, the deterministicvalues for determining membership are as follows:

BLACK_%=the percentage of pixels with grey value less than a blackthreshold value;

WHITE_%=the percentage of pixels with grey value greater than a whitethreshold value;

Sij=Sum of the absolute values of the Laplacians in a window around thepixel being classified;

Range=Max grey Level−Min grey level inside a window around the pixelbeing classified; and

Freq=Measurement of local 2-D frequency around the pixel beingclassified.

To determine the membership value in a particular class, these valuesare compared to a variety of predetermined thresholds in a similarmanner as described above with respect to the three class system. Thevarious classes in this preferred embodiment are demonstrated by thefollowing rules:

If (Sij is >SIJ_HALFTONE and RANGE is >RANGE_HALFTONE and FREQis >FREQ_HALFTONE), then C1 is HALFTONE;

If (Sij is >SIJ_EDGE and RANGE is >RANGE_EDGE and FREQ is<FREQ_HALFTONE), then pixel is EDGE;

If (Sij is <SIJ_HALFTONE and FREQ is <FREQ_HALFTONE), then C1 isCONTONE;

If (C1 is CONTONE and BLACK_% is >BLACK_THRESHOLD), then pixel is BLACK;

If (C1 is CONTONE and WHITE_% is >WHITE_THRESHOLD), then pixel is WHITE;

If (C1 is CONTONE and BLACK_% is <BLACK_THRESHOLD and WHITE_%is >WHITE_THRESHOLD), then pixel is PICTORIAL;

If (C1 is HALFTONE and FREQ is <LOW_FREQ), then pixel isLOW_FREQ_HALFTONE;

If (C1 is HALFTONE and FREQ is >LOW_FREQ and FREQ is <HIGH_FREQ), thenpixel IS MID_FREQ_HALFTONE; and

If (C1 is HALFTONE and FREQ is >HIGH_FREQ), then pixel isHIGH_FREQ_HALFTONE; and

If (pixel is not BLACK and pixel is not WHITE and pixel is not PICTORIALand pixel is not EDGE and pixel is not LOW_FREQ_HALFTONE and pixel isNOT MID_FREQ_HALFTONE and pixel is not HIGH_FREQ_HALFTONE), then pixelis OTHER.

The predicate of each the rules described above is extended to a fuzzytruth instead of an absolute truth to provide the element value for themembership vector. Thus, in order to make the inequality “Y is >X” afuzzy truth, a membership function is defined for “>X”. Similarly, afuzzy membership rule can be defined for <X (very often, the membershipin (<X) is equal to not (>X): (1—membership of (>X)).

For simplicity in implementation, the membership in (>X) is againdefined as follows: $\begin{matrix}{{\mu_{> X}(Y)} = \quad \left( {1,} \right.} & {\quad {{Y > {X + {\Delta \quad X}}},}} \\{\quad {0,}} & {\quad {{{Y <}\quad = {X - {\Delta \quad X}}},}} \\{\quad {{\left( {Y - X + {\Delta \quad X}} \right)/\left( {2\quad \Delta \quad X} \right)},}} & \left. \quad {{{X - {\Delta \quad X}} < Y <}\quad = {X + {\Delta \quad X}}} \right)\end{matrix}$

The value of ΔX determines the level of fuzzification of the class; ifΔX is extremely small, then the definition reduces to the crispdefinition of greater than. It is further noted that although thefuzzification has been described as a linear relationship, the functiondescribing the values between the end points and the mid point may beany type of function. Moreover, the midpoint could represent absolutecertainty in the class and have a membership value of 1 and theendpoints represent absolute certainty of non-membership such that themembership values would graphically form a triangle with the midpointbeing the peak. This type of class could be used for a membershipfunction of the class “=X”.

Returning to the multiple “If's” in the above rules, the membershipvalue of image in the class “edge” would be equal to the fuzzy value ofthe predicate, μ_(edge)(image)=min(μ_(>Slp Threshold)(Σlp),μ_(>Range Threshold) (Max_(grey)−Min_(grey)), μ_(<Freq Threshold)(2DFreq)); the membership value of image in the class “black” would beequal to the fuzzy value of the predicate,μ_(black)(image)=min(μ_(<Slp Threshold) (Σlp), μ_(<Freq Threshold)(2DFreq), μ_(>Black Threshold)(% of black pixels)); the membership value ofimage in the class “white” would be equal to the fuzzy value of thepredicate, μ_(white)(image)=min(μ_(<Slp Threshold) (Σlp),μ_(<Freq Threshold)(2D Freq), μ_(<White Threshold)(% of white pixels));the membership value of image in the class “pictorial” would be equal tothe fuzzy value of the predicate,μ_(pictorial)(image)=min(μ_(<Slp Threshold) (Σlp) μ_(<Freq Threshold)(2DFreq), μ_(<Black Threshold)(% of black pixels), μ_(<White Threshold)(%of white pixels)); the membership value of image in the class “lowfrequency halftone” would be equal to the fuzzy value of the predicate,μ_(lowfreqhalf)(image)=min(μ_(>Slp Threshold) (Σlp),μ_(>Range Threshold)(Max_(grey)−Min_(grey)), μ_(>Freq Threshold)(2DFreq), μ_(<LowFreq Threshold)(2D Freq)); the membership value of imagein the class “mid frequency halftone” would be equal to the fuzzy valueof the predicate, μ_(midfreqhalf)(image)=min(μ_(>Slp Threshold) (Σlp),μ_(>Range Threshold)(Max_(grey)−Min_(grey)), μ_(>Freq Threshold)(2DFreq), μ_(>LowFreq Threshold)(2D Freq), μ_(<HighFreq Threshold)(2DFreq)); and the membership value of image in the class “high frequencyhalftone” would be equal to the fuzzy value of the predicate,μ_(highfreqhalf)(image)=min(μ_(>Slp Threshold) (Σlp),μ_(>Range Threshold)(Max_(grey)−Min_(grey)), μ_(>Freq Threshold)(2DFreq), μ_(>HighFreq Threshold)(2D Freq)).

To implement the fuzzy segmentation process for the two image classsituation, image data received from the image source is divided intoblocks of pixels. The fuzzy image classifier then, to determine amembership value for a particular image type, calculates the summationof the squares of the Laplacians in a window around the pixel beingclassified. Moreover, the fuzzy classifier calculates the percentage ofpixels that have a grey value less than a predetermined black value anddetermines the percentage of pixels within the block which have a greyvalue greater than a predetermined white value.

After calculating this information, the fuzzy classifier determines theconditional value for the condition relating to the summation of thesquares of the Laplacian of every pixel in the block being greater thana Laplacian threshold and the conditional value for the conditionrelating to the sum of the percentage of pixels with a grey valuegreater than the predetermined black value and the percentage of thepixels with a grey value greater than the predetermined white value isgreater than a bi-modal threshold value. Upon determining these twoconditional values, the fuzzy classifier determines the minimal valueand assigns this value as the membership value for the pixel in the“text” class.

The fuzzy classifier then determines the conditional value for thecondition relating to the summation of the squares of the Laplacian ofevery pixel in the block is less than a Laplacian threshold value andthe conditional value for the condition relating to the summation of thepercentage of the pixels with a grey value less than a predeterminedblack value and the percentage of the pixels with a grey value greaterthan the predetermined white value is less the bi-modal threshold value.Upon determining these two conditional values, the fuzzy classifierdetermines the minimal value and assigns this value as the membershipvalue for the pixel in the “contone” class.

The fuzzy classifier thereafter determines the membership value for thepixel in the “other” class by determining the minimal value between1-the “text” membership value and 1-the “contone” membership value andassigns this value as the membership value for the pixel in the “other”class.

In this process, the pixels of image data received by the fuzzyclassifier are assigned membership values for the three possible fuzzyclassifications, “text,” “contone,” or “other.” As noted above, theutilization of the “other” class is necessary in order to avoid havingmembership of the image in all classes to be very small.

An example of implementing fuzzy classification for a laser xerographicprinter, will now be discussed, more specifically, a process, carriedout by a fuzzy classifier, to assign membership values to the pixels ofimage data for eight possible types or classes.

As with the process described above, the process begins by dividing thepixels of image data into blocks of pixels. Thereafter, each block ofpixels is analyzed to determine the sum of the absolute values of theLaplacians in a predetermined window around the pixel being presentlyanalyzed; to determine a range value which is equal to the maximum greylevel minus the minimum grey level inside the predetermined windowaround the pixel being presently analyzed; to determine a frequencyvalue which is equal to the measurement of the local two-dimensionalfrequency around the pixel being presently analyzed; to determine ablack value which is equal to the percentage of pixels that have a greyvalue less than a predetermined black value; and to determine a whitevalue which is equal to the percentage of pixels within the windowhaving a grey value greater than a predetermined white value.

Once these various values are determined, the fuzzy classifier beginsthe assigning of the membership values. The fuzzy classifier determinesthe conditional value for the condition relating to the sum of theabsolute values of the Laplacian in the predetermined window around thepixel being presently analyzed being greater than a halftone threshold,the conditional value for the condition relating to the range valuebeing greater than a range halftone threshold, and the conditional valuefor the condition relating to the frequency value being greater than afrequency halftone threshold. Upon determining these three conditionalvalues, the fuzzy classifier determines the minimal value and assignsthis value as the membership value for the pixel in the “halftone”class.

Then, the fuzzy classifier determines the conditional value for thecondition relating to the summation of the absolute values of theLaplacians in the predetermined window around the pixel being presentlyanalyzed being greater than an edge threshold, the conditional value forthe condition relating to the range value being greater than a rangeedge threshold, and the conditional value for the condition relating tothe frequency value being less than the frequency halftone threshold.Upon determining these three conditional values, the fuzzy classifierdetermines the minimal value and assigns this value as the membershipvalue for the pixel in the “edge” class.

The fuzzy classifier thereafter determines the conditional value for thecondition relating to the sum of the absolute values of the Laplaciansin the predetermined window around the pixel being presently analyzedbeing less than the halftone threshold and the conditional value for thecondition relating to the frequency value being less than the frequencyhalftone threshold. Upon determining these two conditional values, thefuzzy classifier determines the minimal value and assigns this value asthe membership value for the pixel in the “contone” class.

The fuzzy classifier then determines the conditional value for thecondition relating to the pixel being a contone image and theconditional value for the condition relating to the black value beinggreater than a black threshold value. Upon determining these twoconditional values, the fuzzy classifier determines the minimal valueand assigns this value as the membership value for the pixel in the“black” class.

Subsequently, the fuzzy classifier determines the conditional value forthe condition relating to the pixel being a contone image and theconditional value for the condition relating to the white value beinggreater than the predetermined white threshold. Upon determining thesetwo conditional values, the fuzzy classifier determines the minimalvalue and assigns this value as the membership value for the pixel inthe “white” class.

The fuzzy classifier determines the conditional value for the conditionrelating to the pixel being a halftone image and the conditional valuefor the condition relating to the frequency value being less than a lowfrequency threshold value. Upon determining these two conditionalvalues, the fuzzy classifier determines the minimal value and assignsthis value as the membership value for the pixel in the “low frequencyhalftone” class. Thereafter, the fuzzy classifier determines theconditional value for the condition relating to the pixel being ahalftone image, the conditional value for the condition relating to thefrequency value being greater than the low frequency threshold value,and the conditional value for the condition relating to the frequencyvalue being less than a high frequency threshold value. Upon determiningthese three conditional values, the fuzzy classifier determines theminimal value and assigns this value as the membership value for thepixel in the “mid frequency halftone” class.

The fuzzy classifier determines the conditional value for the conditionrelating to the pixel being a halftone image, the conditional value forthe condition relating to the frequency value being greater than thehigh frequency threshold value. Upon determining these two conditionalvalues, the fuzzy classifier determines the minimal value and assignsthis value as the membership value for the pixel in the “high frequencyhalftone” class.

Lastly, the fuzzy classifier determines the membership value for thepixel in the “other” class by determining the minimal value between1-the “edge” membership value, 1-the “black” membership value, 1-the“white” membership value, 1-the “pictorial” membership value, 1-the “lowfrequency halftone” membership value, 1-the “mid frequency halftone”membership value, and 1-the “high frequency halftone” membership valueand assigns this value as the membership value for the pixel in the“other” class.

By utilizing these processes, the fuzzy image classifier can eliminateproblems with class switching in areas between two or more pre-definedtypes. In other words, these processes implement a fuzzy classificationscheme which allows the defining of various image types or classes tofail gracefully. Moreover, by implementing a fuzzy classificationprocess, the fuzzy process allows the fuzzy memberships to haveenhancement, different than any of the pre-defined classes. Morespecifically, the image processing system can choose between a continuumof possible processing operations to produce an output.

As noted above, fuzzy classification avoids sudden transitions in theclassification domain, which results in minimizing abrupt transitions indensity. A practical implementation of fuzzy classification is onlystep-wise continuous (not truly continuous due to the discreteness ofthe data being analyzed). Thus, if the fuzzy classification is overlittle rectangles, the slight residual transitions will still be visibleand objectionable in the rendered output image. On the other hand, apixel-by-pixel without fuzzy classification will not have any visibleartifacts—but will be very noisy or grainy. The transitions will be onan individual pixel basis, but the transitions will be perceived ashuge. Such transitions would be equivalent to injecting huge amounts ofnoise into the image.

However, as contemplated by the present invention, by combining fuzzyclassification with a pixel-by-pixel classification implementation, theoverall classification system can prevent rectangular artifacts (usingpixel-by-pixel classification) in the rendered output image and reducethe classification switching noise (using fuzzy classification). Inother words, the present invention utilizes a fuzzy classificationprocess wherein this process is applied on a pixel-by-pixel basis andnot on a window basis or globally.

In other words, a window implementation would make a classificationbased on the contents of the window and assigned all pixels within thatwindow the same image type classification value or classificationmembership vector value, whereas a pixel-by-pixel implementationclassifies each pixel individually based on the contexts surrounding thepixel or the image values in a neighborhood of this pixel., Although itis desirable for the pixel to be centered, it is not necessary. Forexample, the pixel being classified can be in the lower right of thecontext. In the pixel-by-pixel implementation the neighborhood movesalong the fastscan direction one pixel at a time and along the slowscandirection one scanline at a time, while the window based implementationmoves along the fastscan direction to the next adjoining non-overlappingwindow of pixels and moves along the slowscan direction to the nextadjoining non-overlapping window of pixels.

Another practical advantage of a pixel-by-pixel basis is that the imagecan be classified and rendered in one pass. A windowed or global basedapproach requires the system to analyze the entire region (window orimage) before making a classification decision. By then the video may begone. The system either has to store it, then process it through therenderer beginning after the system has analyzed the entire region, orarrange for the image source to feed through the system twice. Thisincreases system cost and may also slow down the system to ½ throughput.Pixel based requires only a small context since the context is limitedand always in the neighborhood of the pixel being classified.

The present invention also allows the determination of output parameterssuch as filter coefficients and screening level given a fuzzyclassification vector and a corresponding desired output for eachprototype class. The present invention can also accommodate output typesthat must satisfy certain constraints (such as the mean gain of thefilter) through the use of the “other” class. The fuzzy image processinggreatly attenuates switching noise common in crisp decision imagesegmentation processes.

Processing image data for laser printer reproduction requires theimplementation of two separate actions: image processing to enhance,through manipulation, which does not result in loss of information, suchas filtering and TRC translation; and a lossy conversion of theresultant image to a representation accepted by a print engine, normallya reduction in the number of bits per pixel through either applicationof a screen or error diffusion. By utilizing a fuzzy classificationscheme, the present invention can easily unify the processing implied byall of the partial memberships into a single processing action.

For example, each of the classes, defined using fuzzy classification, isprovided with an ideal processing scheme. This ideal image processingscheme would represent the image processing techniques used if themembership in that class was one (1.0) and all other classes had amembership of zero (0.0). The fuzzy image classification process,described above, provides the mechanism for determining the output for agiven fuzzy membership because the output is a vector. Thus, fuzzyclassification can be used to choose each element of the vectorindependently, thereby, as noted above, providing a continuum ofpossible image processing operations. However, this continuum is notdirectly conducive to determining screening operations and determiningfilter coefficients. More specifically, a screening operation, by itsnature, is discrete; either a screen is applied or it isn't. At everypoint, a comparison is made between the image data (video level) and thescreen value (screen level) to determine if a pixel is to be turned OFFor ON. Thus, on its face, screening would appear to be a poor candidatefor fuzzy classification control because of its inability to be appliedat differing levels.

The other problem arises with determining filter coefficients. Imageprocessing requires that the filter coefficients sum to 1 (i.e., aconstant grey level in produces the same grey level out). If there isthe choice between several filters, their combination may no longer meetthis requirement.

The present invention resolves the first situation, the problem withapplying a screen in a fuzzy classification environment, by replacingthe screening operation with a linear operation that provides the sameeffect. Instead of comparing all video to a pre-defined screen, a 2-Dsinusoidal type function, with a maximum amplitude at 45 degrees, isadded to the incoming video.

This screen, referred to as hybrid screening, creates video that is morelikely to cluster output pixels in an error diffusion process. Thisclustering of video is the effect that is very pleasing to the human eyein constant or near constant grey areas. It is the clustering effectthat makes this screened output superior to a straight non-screenederror diffused output.

The frequency characteristics of the 2-D hybrid screen determines theapproximate line pair per inch (lpi) dot pattern seen in the output;however, the ability to collect pixels into large dots, the desiredeffect in a screening process, is dependent on the screen's amplitude.This amplitude, a scalar, can be modified easily using the fuzzy imageclassification process of the present invention. All of the rules canassign a level (i.e., amplitude) of screening (large, medium, small ornone, etc.). The size and frequency of the screen are predetermined tomatch the characteristics of the printer.

The image processing of an image using a fuzzy classification process isvery similar to the fuzzy classification process itself. For example, inthe screening case, the fuzzy image processing method would establishthree screening classes (large, medium, and none) to determine thescreen value to be applied to pixel. Each of these classes would have aset of rules as in the fuzzy classification method to determine themembership value for the pixel in the screening processing class. Themembership value would then be used to calculate the actual screenamplitude which will be described in more detail below.

In a preferred embodiment of the present invention, the rules for thescreening classes are as follows:

If (pixel is “edge” or pixel is “low frequency halftone” or pixel is“mid frequency halftone” or pixel is “other”), then screen is NO_SCREEN;

If (pixel is “black” or pixel is “white”), then screen is MEDIUM_SCREEN;or

If (pixel is “pictorial” or pixel is “high frequency halftone”), thenscreen is FULL_SCREEN.

Referring to the multiple “If's” in the above rules, the membershipvalue of image in the class “NO_SCREEN” would be equal to the fuzzyvalue of the predicate, μ_(NO) _(—)_(SCREEN)(screen)=max(μ_(edge)(pixel), μ_(lowfreqhalftone)(pixel),μ_(midfreqhalftone)(pixel), μ_(other)(pixel)); the membership value ofimage in the class “MEDIUM_SCREEN” would be equal to the fuzzy value ofthe predicate, μ_(MEDIUM) _(—) _(SCREEN)(screen)=max(μ_(black)(pixel),μ_(white)(pixel)); and the membership value of image in the class“FULL_SCREEN” would be equal to the fuzzy value of the predicate,μ_(FULL) _(—) _(SCREEN)(screen)=max(μ_(pictorial)(pixel),μ_(highfreqhalftone)(pixel)).

To implement the fuzzy screening, the processing system determines themembership value of the pixel in each of the classes associated with aparticular screening process and assigns the maximum value as themembership value in the screening class. For example, if the pixel hadmembership values for “edge,” “low frequency halftone,” “mid frequencyhalftone,” and “other” of 0.6, 0.7, 0.2, and 0.3, respectively, theprocessing system would decode the membership vector and assign amembership value to the NO_SCREEN class of 0.7. Moreover, if the pixelhad membership values for “black” and “white” of 0.6 and 0.5,respectively, the processing system would decode the membership vectorand assign a membership value to the MEDIUM_SCREEN class of 0.6. Lastly,if the pixel had membership values for “pictorial” and “high frequencyhalftone” of 0.3 and 0.4, respectively, the processing system woulddecode the membership vector and assign a membership value to theFULL_SCREEN class of 0.4.

To determine the actual amplitude for the screen value, the fuzzyprocessing module, in the preferred embodiment of the present invention,multiply the membership value for each screen class with the idealcoefficient for that class and divide the product by the sum of themembership values. In the preferred embodiment, the NO_SCREENcoefficient is 0, MEDIUM_SCREEN coefficient is 0.5, and FULL_SCREENcoefficient is 1.0. Thus, in the example described above, the screenamplitude for the pixel being processed would be equal to a scalar valueof 0.412 (((1.0*0.4)+(0.5*0.6)+(0.0*0.7))/(0.4+0.6+0.7)).

An example of the application of fuzzy classification to screening isillustrated in FIG. 8. As illustrated in FIG. 8, the video signal orimage data is fed into a fuzzy classifier 80 which classifies the imagedata according to the rules described above. The fuzzy image classifier80 then generates a membership vector which is passed onto a screeninggenerating circuit 88. The screen generating circuit 88 produces ascreen value which is added to the image data at adder 89. The imagedata which is summed with the screen value corresponds to the same pixelas was classified by the fuzzy image classifier 80. In other words, thesystem illustrated in FIG. 8 also includes buffers (not shown) to insurethat the pixel being processed corresponds to the correct membershipvector being used to define the processing parameters.

A more detailed example of the application of fuzzy segmentation toscreening is illustrated in FIG. 9. As illustrated in FIG. 9, the videosignal or image data is fed into a fuzzy classifier 80 which classifiesthe image data according to the rules described above. The fuzzy imageclassifier 80 then generates a membership vector which is passed onto ascreening generating circuit 88. The screen generating circuit 88includes a screening weighting circuit 883, a screen value generatingcircuit 881, and a multiplier 885.

The screen weighting circuit 883 generates a weighting factor inresponse to the values in the membership vector so as to produce a noscreen, a medium screen, or a high screen, or any screen therebetween,as discussed above. In other words, if the pixel had membership valuesfor “edge,” “low frequency halftone,” “mid frequency halftone,” and“other” of 0.8, 0.7, 0.1, and 0.3, respectively, the processing systemwould decode the membership vector and assign a membership value to theNO_SCREEN class of 0.8. Moreover, if the pixel had membership values for“black” and “white” of 0.4 and 0.1, respectively, the processing systemwould decode the membership vector and assign a membership value to theMEDIUM_SCREEN class of 0.4. Lastly, if the pixel had membership valuesfor “pictorial” and “high frequency halftone” of 0.2 and 0.0,respectively, the processing system would decode the membership vectorand assign a membership value to the FULL_SCREEN class of 0.2.

Thus, in this example, the screen amplitude for the pixel beingprocessed would be equal to a scalar value of 0.286(((1.0*0.2)+(0.5*0.4)+(0.0*0.8))/(0.2+0.4+0.8)).

The screen value generating circuit 881, which may be a lookup table ora hardwired circuit, produces a screen value based on the position ofthe pixel (image data) within the image. The weighting factor from thescreen weighting circuit 883 and the screen value from screen valuegenerating circuit 881 are multiplied together by multiplier 885 toproduce the screen value to be added to the pixel. This screen value isadded to the image data at adder 89.

The image data which is summed with the screen value corresponds to thesame pixel as was classified by the fuzzy image classifier 80. In otherwords, the system illustrated in FIG. 9 also includes buffers (notshown) to insure that the pixel being processed corresponds to thecorrect membership vector being used to define the processingparameters.

As noted above, the problem of using a fuzzy image classification systemto control digital filtering is two-fold. First, digital filtering isnot a scalar function, but a matrix function. Secondly, there is aconstraint on the matrix function; the filter must have gain of 1.0 at(ω₁ω₂)=(0,0). This ensures that constant grey areas are not altered byfiltering.

To solve this problem, the present invention, in the preferredembodiment, uses the weighted summation of several pre-defined filtersto produced the filtered results. These filters are filters associatedwith a particular filter class; i.e., one filter class is enhance, onefilter class is lowpass, and another filter class is “other”. Thedigital filter of the present invention takes the form ofF_(o)=Σμ_(i)F_(i) where all of the F_(i)s correspond to the filtersassociated with each i^(th) class (or classes) and μ_(i) corresponds tothe membership of the image in the ith class, as determined by the fuzzyprocessing routine.

In a preferred embodiment of the present invention, the rules for thefiltering classes are as follows:

If (pixel is “edge” or pixel is “pictorial” or pixel is “low frequencyhalftone”), then filter is ENHANCE; or

If (pixel is “high frequency halftone”), then filter is LOWPASS.

Referring to the multiple “If's” in the above rules, the membershipvalue of image in the class “ENHANCE” would be equal to the fuzzy valueof the predicate, μ_(ENHANCE)(filter)=max(μ_(edge)(pixel),μ_(lowfreqhalftone)(pixel), μ_(pictorial)(pixel)); and the membershipvalue of image in the class “LOWPASS” would be equal to the fuzzy valueof the predicate, μ_(LOWPASS)(filter)=max (μ_(highfreqhalftone)(pixel)).

To implement the fuzzy screening, the processing system determines themembership value of the pixel in each of the classes associated with aparticular screening process and assigns the maximum value as themembership value in the screening class. For example, if the pixel hadmembership values for “edge,” “low frequency halftone,” and “pictorial”of 0.6, 0.7, and 0.3, respectively, the processing system would decodethe membership vector and assign a membership value to the ENHANCE classof 0.7. Moreover, if the pixel had membership values for “high frequencyhalftone” of 0.6, the processing system would decode the membershipvector and assign a membership value to the LOWPASS class of 0.6.

To determine the actual coefficients for the various filters, the fuzzyprocessing system must ensure that the fuzzy filtering, resulting fromthe rule set, meets the constraint of a gain of 1.0 at the frequency(ω₁ω₂)=(0,0). To alleviate this problem, one of the output filterchoices is assigned the bypass function. In bypass, Vout=Vin; i.e.; nofiltering is done. Thus, the resulting filter, according to the conceptsof the present invention is F_(o)=Σμ_(i) F_(i)+(1−Σμ_(i)) * F_(B) suchthat when the desired effect is the bypass filter, the value goes tozero and the effect of the filter F_(i) is ignored.

It is noted that the enhancement filter amplifies all higherfrequencies, and the lowpass filter attenuates the higher frequencies.The coefficient value, c, for the bypass filter can be determined usingc_(bypass)=1−μ_(enhance)(filter)−μ_(lowpass)(filter) so that the outputfilter can be described as F_(o)=μ_(enhance)(filter) *F_(enhance)+μ_(lowpass)(filter) * F_(lowpass)+c_(bypass)* F_(bypass).

An example of image processing an image using fuzzy classification withrespect to filtering the image data is illustrated in FIG. 7. Asillustrated in FIG. 7, video data or image data is fed into a fuzzyclassifier 80, a lowpass filter 81, and an enhanced filter 82, and abypass filter 83 in parallel. As described above, the fuzzy classifier80 determines the membership vector of the pixel to be processed by theparallel set of filters. Note that this process includes buffers (notshown) to insure that the pixel being filtered corresponds to thecorrect membership vector being used to define the processingparameters.

Based upon this classification, the fuzzy classifier 80 will cause theoverall filter to generate a set of coefficients which are applied tomultipliers 84, 85, and 86. The coefficients will enable the overallfilter to weight the output from the various filters according to thefuzzy image classification.

For example, as noted above, if the pixel had membership values for“edge,” “low frequency halftone,” and “pictorial” of 0.6, 0.7, and 0.3,respectively, the processing system would decode the membership vectorand assign a membership value to the ENHANCE class of 0.7, which in turnis the filter coefficient assigned to the enhance filter 82 and fed tomultiplier 85. Moreover, if the pixel had membership values for “highfrequency halftone” of 0.6, the processing system would decode themembership vector and assign a membership value to the LOWPASS class of0.6, which in turn is the filter coefficient assigned to the lowpassfilter 81 and fed to multiplier 84. This leaves the generation of thecoefficient for the bypass filter 83.

As noted above, the generated coefficients need a relationship such thatthe sum of the coefficients will not be greater than 1 so as to keep thegain of the overall output from the filters to be equal to 1. Thus, inthe preferred embodiment of the present invention, the coefficient forthe bypass filter is 1 minus the enhance coefficient minus the lowpasscoefficient (in the example, 1−0.7−0.6=−0.3). This coefficient isapplied to the multiplier 86. The weighted filter outputs are then fedinto an adder 87 which adds all the outputs to produce a filtered pixelof image data which has been filtered according to its fuzzy imageclassification.

Although FIG. 7 illustrates the utilization of a fuzzy classifier with aplurality of different filters, the fuzzy classifier can also beutilized in connection with a modular time variant two-dimensionalnon-separable filter wherein the non-separable filter is made up of aplurality of separable filters.

The utilization of separable filters allows a non-separable filter of aparticular form to be implemented in a cost effective manner. Moreover,a separable two-dimensional filter can be described as one-dimensionalfilter that acts in the vertical direction or slowscan directionfollowed by another one-dimensional filter acting in the horizontal orfastscan direction. Thus, the filter matrix can be described as theproduct of F_(vh)=f_(v)*f_(h) wherein F_(vh) is an N by M matrix, f_(v)is a N length column vector (the vertical filter) and f_(h) is a Mlength row vector (the horizontal filter).

If the matrix F_(vh) cannot be represented using the above equation, thematrix is considered non-separable and the separable filterimplementation cannot be used. However, if the N by M matrix isrepresented using a singular value decomposition such as F_(vh)=U*S*V,where U is N by N unitary matrix, S is an N by M diagonal matrix, and Vis a M by M unitary matrix; a separable filter implementation can beused. Furthermore, if N and M are not equal, the above equation can bealtered to F_(vh)=U_(r)*S_(r)*V_(r), where Q=min(n,m), U_(r) is a N by Xsubmatrix of U, S_(r) is a Q by Q submatrix of S, and V_(r) is a M by Qsubmatrix of V. Putting this in summation form, the resultingrepresentation will be F_(vh)=Σs(i)*u_(i)*v_(i), where i is greater thanor equal to 1 but less than or equal to Q.

In this representation, the vector s(i) is a diagonal of the matrixS_(r), u_(i) is the i^(th) column vector of U_(r), and v_(i) is thei^(th) column vector of V_(r). Each component is a separable filtersimilar to that described above with the exception of a gain factors(i). In other words, any non-separable filter of length N by M can berepresented as the weighted summation of Q separable N by M filters.Thus, to implement a non-separable filter using the weighted averages ofseveral of the filters, the hardware becomes a conglomeration of Qseparable filter modules, Q multipliers, and an adder.

Although the above described fuzzy classification system resolves someof the problems with pixel classification, the method described toimplement this system does not necessarily provide flexibility normaximum efficiency. What is desired is a segmentation architecture whichallows flexibility and efficiency such that the rules for classificationcan be readily changed in the system as new information is uncoveredabout the nature of printed documents or a user desires to override acertain process on the fly.

One embodiment of the present invention provides an architecture whichprovides the above features. This architecture consists of imposing arigid partitioning of the segmentation process into three parts. Abetter understanding of this concept can be seen in the illustration ofFIG. 10.

FIG. 10 illustrates a block diagram of a flexibleclassification/segmentation system. As illustrated in FIG. 10, the videosignal or image data is fed into a microclassifier system 100, amacro-reduction system 200, and a pixel classification look-up table300. The microclassifier system 100 is made of a plurality ofmicroclassifier circuits 101-104. Microclassifier circuits measureintrinsic properties of the video extracted through a simplemathematical formula or heuristic. Examples of microclassifiers might bedetected video minimums or maximums, Laplacian, Sum of Laplacians,Average, Range of Raw Video, Range of Average Video, etc. as used in allof the references. The microclassification values are fed into themacro-reduction system 200.

The many microclassifiers in this embodiment have different magnitudeswhen computed for images of different types. Table 1 lists qualitativelythe magnitude of the microclassifiers for different image types.

TABLE 1 Halftone Halftone Text Contone Contone Contone Gray Shadow EdgeGray White Black Video Varying Varying Varying Varying High Low PeakCount Frequency Varying Near Near Near Zero Near Zero Dependent ZeroZero Sum of High Low Highest Low Low Low absolute Laplacian in X × Y*Range of Highest High Highest Low Low Low Video in 5 × 5 *X = 7, 11 andY = 3, 5 for resolution of 400 spi and 600 spi, respectively.

The magnitude of the responses of the microclassifiers shown abovesuggests the following microclassification rules expressed in a pseudocode.

Vij = Video of the j-th pixel in the i-th scanline. Nij = Peak Countcentered at the j-th pixel in the i-th scanline. Sij = Sum of AbsoluteLaplacian of Video at the j-th pixel in the i-th scanline. Rij = Rangeof Video in 5x5 context centered at the j-th pixel in the i-th scanline.N16 > N15 >...>N1 = Thresholds in Peak Counts for Halftones of classeswith frequencies HFT16 > HFT15 >...HFT1 S3 > S2 > S1 = Thresholds inSij. R1 = Threshold in Rij. V2 > V1 = Thresholds in Vij. If (Nij >= N1AND Sij >= S2 AND Rij >= R1 { If (Nij < N2) Class = HFT1 else if (Nij <N3) Class = HFT2 else if (Nij < N4) Class = HFT3 else if (Nij < N5)Class = HFT4 else if (Nij < N6) Class = HFT5 else if (Nij < N7) Class =HFT6 else if (Nij < N8) Class = HFT7 else if (Nij < N9) Class = HFT8else if (Nij < N10) Class = HFT9 else if (Nij < N11) Class = HFT10 elseif (Nij < N12) Class = HFT11 else if (Nij < N13) Class = HFT12 else if(Nij < N14) Class = HFT13 else if (Nij < N15) Class = HF14 else if(Nij <N16) Class = HFT15 else Class = HFT16 } else if (Sij >= S3) { Class =EDGE } else { if(Vij < V1) Class = BLACK else if(Vij >= V2) Class =White else { if(Sij >= S1) Class = CONTONE ROUGH else Class = CONTONESMOOTH } }

In the microclassification rules shown above, the halftone class isdivided into 16 frequency intervals as an example. The multiple halftoneclasses allow an adaptive filtering and rendering of the halftone areas.It eliminates classification artifacts resulting from simply dividingthe halftone classes into high and low frequency halftones only. Thecontinuous tone is also divided into two sub-classes, smooth and rough,based on the sum of the absolute Laplacian. This allows for moreoptimized rendering of the continuous tone area. More specifically, themultiple frequencies are used to realize the advantages of a true fuzzyclassification system through a discrete implementation. The subclassesare created as a way to gradate the overall classification, thusreducing the need for fuzzy classification. In a classification fuzzysystem, there would be fewer classes, for example low, medium and highfrequency.

A variant of the above microclassification rules is obtained byreplacing the statement “else if(Sij >=S3)” with “else if(Sij>=S3 ANDRij>=R1)” to obtain a more robust determination of the edge classmembership value.

The macro-reduction system 200 is made of a plurality of macro-reductioncircuits 201-204. Each macro-reduction circuit reducesmicroclassification values received from the microclassifiers to producehigher level, more directly useful information through mathematicaloperations and heuristics, suppressing noise and other irrelevant orundesirable variations in the microclassifiers. These are oftennon-linear operations. Examples of macro-manipulations or reductionsmight be to count video minimums or maximums in the neighborhood of apixel, implementing a running average with a controllable attack, decay,and heuristic conditions, performing an auto-correlation on detectedpeaks, or combining two or more microclassifiers to compensate forundesirable fundamental effects. The macro-reduced values are fed intothe pixel classification look-up table 300.

The pixel classification look-up table 300 reduces the macro-reducedoutputs to the final classification through a run-time programmablemechanism. The look-up table 300 also enables any arbitrarily complexfunction to be performed, since the arbitrarily complex function can bepre-calculated off-line and loaded into the table. Microclassificationvalues or even the input video may also be used to directly drive theprogrammable table 300 if appropriate. Use of a programmable tableenables quick modification of the classification algorithm whenrequirements change or as understanding the properties of images grows,albeit within the framework presented at the input of the table.

This architecture can be further modified to add more flexibility asillustrated FIG. 11. As illustrated in FIG. 11, a programmable logicarray 400 is placed between the microclassifier system 100 and themacro-reduction system 200. The programmable logic array 400 includes aplurality of circuits, gates, and multipliers which is programmed tocontrol which microclassification value goes to which macro-reductioncircuit and what is the weight of that value. This programmable logicarray 400 enables the user or technician to reprogram the overallsegmentation algorithm by allowing for the programmability of thefeeding of the microclassification values to the macro-reductioncircuits. Such a programmable logic array could also be included betweenthe macro-reduction system 200 and the programmable look-up table 300 toadd maximum flexibility.

In any of the architectures described above with respect to FIGS. 10 and11, each stage can be easily scaled and interactions added or removed asrequirements change. Microclassifiers can also be removed to save costor added to extract new kinds of information from the document.Macro-reductions can be removed to save cost or added when newinterrelationships are discovered or better methods of reducing themicroclassifier outputs are developed. The lookup table can be expandedto bring a higher level of programmability up to the user or reduced tosave cost. Lastly, such architectures enable classifications to bedynamically added, removed, or their defining attributes modified bothbetween and within pages of a multiple page image processing job.

As noted above, various types of information about the image is neededto provide a proper image type classification of the image data. Theinformation varies from determining the peaks in the video stream todetermining the sum of the absolute values of the Laplacian of the videostream. One important tool in determining some of the needed informationis filtering. Filtering has been utilized in various segmentationsystems. Such filtering is essential in the measurement or estimation ofthe halftone frequency of the video signal or image data.

As noted above, methods for classifying image pixels according todocument types; e.g., text, continuous tone, and halftones of varioustechniques and frequencies; make decisions based on image context in thevicinity of the pixel being classified. A critical piece of informationextracted from the context is the estimate of the halftone frequency.The frequency is commonly estimated from a count of halftone peakswithin the context. Such a count is really simply the application of atwo-dimensional blur filter with rectangular amplitude profile in thespace domain.

One embodiment of the present invention is directed to an efficient andinexpensive method for implementing a triangular blur filter to produceeither a low pass or a high pass filter. The high pass filter is derivedby the usual conventional method of subtracting the low pass output fromthe original input. The implementation of the present invention requiresonly two adders and two delay elements and no multipliers. In addition,the present invention provides for inexpensively re-synchronizing data,which is delayed due to the filtering, with any associated input tagswherein tags can be any auxiliary data associated with the image data orvideo stream, such as image classification information.

In implementing the filter, the present invention redundantly performsthe calculation on the trail edge instead of storing the result to useit later. This is only done in the slowscan direction. In the fastscandirection, the results are stored in a series of latches. The reasoningbehind this implementation will become more clear from the descriptionbelow.

In the fastscan direction, the “calculation” is that of taking grayinput video and detecting peaks, yielding a 1-bit output.Conventionally, it is a very expensive operation because it requires alarge number of gates. On the other hand, it is inexpensive to delaythis one bit output by a few pixels (8-18) with flip-flops, so it isdelayed. In the slowscan direction, the “calculation” is a peak counterwithin an one-dimensional window, inputting a 1-bit number and yieldinga 4-bit number. This 4-bit result would have to be delayed 8-18scanlines, times some 5000 pixels per scanline, costing several hundredthousand flip-flops, or more likely an external RAM and controlcircuitry.

In view of this inefficiency, the present invention duplicates thefirst-pass block to recreate the result when and where needed during thesecond pass rather than using the conventional method of taking theoutput of the first pass and providing it as input to the second pass astwo pure blocks.

For many different applications, it is desirable to filter a given inputsignal or image. Typically for signal and image processing applications,finite impulse response filters are preferable because of severalproperties these filters possess, specifically, linear phase andguaranteed stability. In addition to the input/video that is to befiltered, additional properties/data associated with the inputsignal/video may be coupled with the signal/video as it flows throughthe system. This additional information may be processed by or controldownstream modules. This additional information often containsinformation corresponding to the classification of the individual pixelsof the signal/video, such as membership information, and may be referredto as the signal/image tag or a classification vector. In thedescription below, the tag at a certain time/position k will be referredto as T(k). Thus, the entire input to be processed is the union of boththe signal/video plane and one or more tag or classification vectorplanes.

For the application of filtering, it is desirable to filter thesignal/video input and then take the corresponding output and reunite itwith the appropriate tags. An example of such an application offiltering can be expressed utilizing the following equation for arectangular filter of length 2N+1.

H(z)=Σ_(i) z ⁻¹ , i={−N, −N+1, . . . 0, . . . N−1,N}

y(k)=Σ_(i) x(k+i), i={−N, −N+1, . . . 0, . . . N−1,N}

In these equations, H(z) is the z-transform of the filter and x(k) andy(k) are the inputs and outputs of the filter, respectively. Althoughthis filter has poor frequency response characteristics, it is oftenused because it is very inexpensive to implement.

In order to implement a rectangular filter in real time andinexpensively, the following rectangular filter is conventionallyutilized.

H(z)=(1−Z^(−2N−1))/(Z^(−N)−Z^(−N−1))

y(k−N)=y(k−N−1)+x(k)−x(k−2N−1);

To get the present sum from the above equation, the previous sum is usedand one video input is subtracted while another is added. In addition,none of the terms H(z) are functions of positive exponents of z and thusthey do not depend on future values of x, only on the present andprevious values. This type of rectangular filter makes it easy toimplement in real time and/or in pipeline systems where all of the datais not available at once but is presented serially to the filter. Unlikea typical filter, which requires several values of the input and severalmultipliers/adders, all that is needed to implement this rectangularfilter is a delay element of 2N+1 and two adders.

Unfortunately, it is noted that at time (or position) index k, theoutput is known up to time k−N. At this time however, the tag orclassification vector is at time k, T(k) is the value available in thepipeline. In other words, the output video and the associated tag orvector have become unsynchronized. In order to re-synchronize the outputsignal/video with the proper tag, the tags must be delayed by N sampletimes. The overall configuration therefore requires one signal to bedelayed by N (the tags), and another signal be delayed by 2N+1 (thevideo). If expensive first-in first-out (FIFO) buffers are utilized forthese delays (as in the case of two-dimensional image processing), theexpense of the two delay blocks can be very large. In addition, such animplementation would require two different lengths of FIFO buffers ortwo identical FIFO buffers controlled differently.

Furthermore, a single rectangular filter is not the optimal filter toutilize because its side lobes in the frequency domain are verysignificant. Thus, to create a better filter, one embodiment of thepresent invention utilizes a rectangular filter that is usedsuccessively to provide an equivalent triangular filter wherein thetriangular filter is generated by the convolution of two rectangularfilters. Such a triangular filter can be expressed utilizing thefollowing equations.

z(k−2N−1)=z(k−2N−2)+y(k−N−1)−y(k−3N−2);

y(k−N)=y(k−N−1)+x(k)−x(k−2N−1);

In the above equations, y(k−N) is a first state sum as in a rectangularfiltering and z(k−2N−1) is the final output from the triangular filter.This triangular filter equation can be reduced to:

z(k)=Σ(2N+1−|i−2N−1|)x(k+i−2N−1), i=0 to 4N+1

The triangular filter has superior frequency properties when compared toa rectangular filter as clearly illustrated in FIG. 21. FIG. 21 showsthe Fourier transform of both filters when each is 15 elements long. Itis noted that to implement this triangular filter, the triangular filterrequires four adders/subtractors. In addition, the triangular filterrequires the use of two delay blocks instead of one but both have afixed delay of 2N+1. If a single delay block is wide enough, it can beused to delay both the tags/vectors and the signal. In other words,instead of two delay blocks, a single block of twice the bit width cansuffice.

Additionally, the triangular filter has the property that the overalldelay between the input and output is 2N+1, unlike the delay of N in therectangular filter. Thus, utilizing a triangular filter requires thatthe tags t(k) be delayed by 2N+1 to be re-synchronized with the outputvideo. This delay is identical to the delay needed for the input signalx(k), an intermediate state y(k) in the equation above describing thetriangular filter. Therefore, the same type of delay block can be usedfor the image elements as well as the tags. If the wordlength of thedelay block is large enough, a single delay block can be used to delayx(k), y(k), and t(k). The triangular filter can be implemented usingonly a single delay element of 2N+1 states with four adders andsubtractors. In addition, one multiplier can be used to normalize thefilter so that it is low pass in nature with a unity gain. Thenormalization can also be done at the end of the first state: i.e., ony(k) to reduce the bit to be carried into the second state sum.

With only minimal additional hardware, a rectangular filter can beeasily replaced by a triangular filter, which has better frequencyresponse characteristics. Moreover, the tags in the signal can bereadily re-synchronized using common hardware to that which is needed bythe filter itself. In addition, the triangular filter can be used toimplement a high pass filter as well. The signal/video input can bedelayed along the tag vector so that at every index k there is theoriginal signal, the low pass signal (via the triangular filter) and thetags. The high pass filter output is easily created by subtracting thelow pass output from the original input video.

This process can be repeated to provide “smoother” low pass filterswherein the filter becomes Gaussian as the procedure is infinitelyrepeated. The cost per stage is linear, but the incremental benefits ofthe low pass filter and the frequency response decrease rapidly witheach additional stage.

Finally, the process discussed above can be extended to two dimensionsby using a triangular filter in the horizontal direction and atriangular filter in the vertical direction. An example of a typicalapplication of a two-dimensional filter is the utilization of atwo-dimensional rectangular function to estimate the spacial frequency(measured in lines per inch, or LPI) of a halftone image given the peakmap of the image. More specifically, such an estimation of the frequencyis needed in any process requiring automatic halftone detection such asautomatic image segmentation. In such an application, a filter is usedto process a peak map and thereafter each point is converted to afrequency estimate by taking the square root of the peak count within acontext centered on the pixel whose containing halftone cell's frequencyis being estimated and multiplying it by a constant which is a functionof the filter size. An ideal frequency measurement map would have astandard deviation of 0.0 since the input image is a constant halftone.

As discussed earlier, the triangular filter can be implemented as theconvolution of two successive rectangular blur filters. Thisimplementation has the advantage of maintaining synchronization betweenthe video being filtered and any information in a parallel channel, suchas classification tags. Therefore, determining the optimalimplementation of a two-dimensional triangular filter reduces to theproblem of determining the optimal implementation of a two-dimensionalrectangular blur filter. Furthermore, while the desire to minimize costand complexity drives toward filtering of tiled rectangular blocks atfixed locations in the image, this leads to rectangular artifacts in theclassification map, which in turn results in objectionable rectangularartifacts in the resulting image whose processing is driven by this map.This understanding adds the additional requirement that the optimalimplementation of a two-dimensional triangular filter must filter eachpixel based on a context which is centered on that pixel.

The present invention provides a fast, hardware efficient method ofachieving the goals outlined above: [1] Implementing two concurrenttwo-dimensional rectangular blur filter on a raster image flowingthrough a pipeline, yielding a triangular filter, [2] Filtering based ona context centered on each pixel being filtered, thereby avoidingrectangular artifacts, and [3] Maintaining synchronization of thefiltered output image with a unfiltered auxiliary data channel. In theevent that the data needs to be processed through additional rectangularblur filters to obtain better frequency characteristics, only one set ofcontrol circuitry is required for all instances.

To better explain the two-dimensional rectangular blur filter of thepresent invention, FIGS. 22 through 24 will be described below.

FIG. 23 is a block diagram of a circuit which detects the peaks of thevideo stream and counts the number of peaks within a programmableneighborhood around the pixel being classified. The video stream isreceived by a peak detection circuit 600 which detects the peaks withinthe video stream utilizing conventional peak detection methods. Thiscircuit outputs a binary 1 if a peak is detected and a binary 0 if nopeak is detected. The peak detection information is fed into a peakcounter circuit 500 wherein the number of peaks in the contextneighborhood is determined.

The peak counter circuit 500 includes a programmable delay line 602which provides trail edge peak detection information. In other words, todetermine the number of peaks in a neighborhood that is moving as theclassification process goes from one pixel to the next pixel, thecounter must not only count the incoming peak detection information, butmust also remove the peak information relating to pixels now outside theneighborhood. The programmable delay line 602 provides the trail edgepeak detection information which is the peak information relating topixels now outside the neighborhood. The delay line 602 is programmableso that the system can be programmed to implement varying targetneighborhood sizes based on image resolution, apriori knowledge of thedocument being segmented, and system level tradeoffs.

The trail edge peak detection information, along with the current peakdetection information from the peak detection circuit 600, is fed into apeak counter 604 which adds the current peak detection information to acurrent count, provided by a latch circuit 606 and subtracts the trailedge peak detection information provided by the programmable delay line602 from the current count. In this way, the peak counter 604 canprovide an accurate current count to the latch 606.

One way to implement a two-dimensional rectangular blur filter using thepeak detection/counter circuitry of FIG. 23 is illustrated in FIG. 24.The video stream, as illustrated in FIG. 24, is received by a peakdetection circuit 600 which detects the peaks within the video streamutilizing conventional peak detection methods. This circuit outputs abinary 1 if a peak is detected and a binary 0 if no peak is detected.The peak detection information is fed into a fastscan lead edge peakcounter circuit 608 wherein the number of peaks in the contextneighborhood is determined. The fastscan lead edge peak counter circuit608 contains the same components as the peak detection circuit 500 ofFIG. 23.

Since the filtering needs to be done in two dimensions, the peakinformation within a context extending in the slowscan direction mustalso be included in the peak count determination. To facilitate theinclusion of slowscan peak count information, a slowscan trail edge FIFObuffer 610 is included. A slowscan lead edge peak counter is notrequired because the scanning on the image data is along the fastscandirection such that when the scanning moves to the scanline, the peakinformation for that scanline will be provided by the fastscan lead edgepeak counter 608 as it scans across the image data, whereas the slowscantrail edge FIFO buffer 610 will provide the slowscan peak countinformation that needs to be removed from the accumulated count becausethat scanline is no longer part of the neighborhood being analyzed. ThisFIFO buffer 610 must be large enough to store information for each pixelin the fastscan direction (typically on the order of 2,500-10,200pixels) and for the number of scanlines that define a neighborhood.

The peak count information from the fastscan lead edge peak counter 608is fed to a summer 612 and the slowscan trail edge FIFO buffer 610. Thesummer 612 adds the peak count information from the fastscan lead edgepeak counter 608 to a previous count value for that pixel column thatwas stored in a scanline FIFO buffer (not shown) to generate the realcount value for the pixel being classified. This real count value isalso fed into a summer 614 wherein the real count value is modified sothat the peak count information from the slowscan trail edge is removed.This modified value is stored in the same scanline FIFO buffer.

The system in FIG. 24 produces a two-dimensional rectangular blur filterthat eliminates rectangular artifacts induced by the filter, butrequires large amounts of high speed memory, thus having a negativeimpact on its cost. FIG. 22 illustrates another implementation of atwo-dimensional rectangular blur filter which does not require a largeamount of memory and produces identical results.

As illustrated in FIG. 22, peak detection information from a peakdetection circuit (not shown) is received by a latch 501. The latchedinformation is fed to a fastscan trail edge peak counter 503 which hasessentially the same contents as peak counter 500 in FIG. 23. The trailedge count information is fed into a latch 504 and this latchedinformation is fed into a summer 505.

Peak detection information from a second peak detection circuit (notshown) is received by a latch 513. The latched information is fed to afastscan lead edge peak counter 514 which has essentially the samecontents as peak counter 500 in FIG. 23. The lead edge count informationis fed into a summer 510. Summer 510 also receives latched peak countinformation from latches 508 and 509. This latched informationrepresents the old count value that had been stored in a scanline FIFObuffer (not shown). The resulting sum from summer 510 is the currentcount value for the pixel being classified as it leaves the latchingcircuit of latches 511 and 512.

The resulting sum from summer 510 is also fed to summer 505 throughlatch 507 wherein summer 505 subtracts the latched count value of thefastscan trail edge counter 503 from the latched summed value fromsummer 510. This value is latched by latch 506 before being stored inthe scanline FIFO buffer. It is noted that the various latches in FIG.22 are merely for keeping the various counts for the active pixel columnproperly synchronized with the pixel image data, and thus, these latchesdo not go to the crux of the invention since more or less latches may beneeded in order to maintain this synchronization depending on detaileddesign details.

The operations of the two-dimensional rectangular blur filter asillustrated in FIG. 22 will be more precisely described below.

In order to calculate the output of the blur filter for a context trailedge pixel position in both the fastscan and slowscan directions, twofastscan blur filters (one at the slowscan lead edge of the context andone at the slowscan trail edge of the context) feed P slowscan filterswherein P is the number of pixels per scanline in the fastscandirection. The state of the P slowscan blur filters is stored in a ringbuffer and restored, updated, and saved every clock cycle as eachpixel's contribution is incorporated. The details of the process are asfollows:

The filter of the present invention blurs pixels in a 1 pixel slowscanby N pixel fastscan strip as it slides in the fastscan direction at theslowscan lead is edge of the incoming image:

Given: N: Fastscan length of the blur filter

X: Current pixel position on the scanline.

V_(X): Input video value at the current pixel position.

FSPS_(X): Fastscan Partial Sum consisting of the sum of the N−1 inputpixels preceding the current pixel on the scanline.

The blur filter output consisting of the sum of the N pixels trailingand including the pixel at position X is FSBLUR_(X)=FSPS_(X)+V_(X) andthe new partial sum for the next column position is:FSPS_(X+1)=FSBLUR_(X)−V_(X−(N−1)). The N−1 trailing video values aresaved in a FIFO P elements long so that random and repeat access to theimage is not required. At each pixel location V_(X−(N−1)) is immediatelyavailable from the output of the FIFO and V_(X) is written to the input.

An output image consisting of the fastscan blurred video can then bepiped into a similar process to perform the slowscan blurring. Howeversince the data is presented in a fastscan/slowscan raster format inorder to blur each column by M pixels in the slow scan direction, M fullscanlines of (BPP+┌Log₂ N┐) bit wide storage would be required whereinBPP indicates the number of bits per pixel. Instead, the original BPPbit wide input data can be delayed and presented to a second fastscanblur block identical to that described above. This second fastscan blurcan concurrently recalculate the required quantity (FSBLUR_(X,(Y−(M−1)))in the description below) for use by the slowscan blur operation.

In the event that the original BPP bit wide input data is already beingdelayed and presented at the slowscan trail edge for other reasons, thisin effect saves all M scanlines by (BPP+┌Log₂ N┐) bits of storage. Forexample, if BPP=1 and N=16, 5 bits wide by M scanlines of storage aresaved. If multiple instances of the rectangular filter are applied,┌Log₂ N┐ bits by M scanlines of storage are saved over the (BPP+┌Log₂N┐) required where the fastscan blurred video is simply piped to asecond stage. This process is described in detail below.

For each column in the fastscan blurred image, the filter of the presentinvention blurs a M pixel slowscan by 1 pixel fastscan strip as itslides in the slowscan direction at the slowscan lead edge of theincoming image.

Given:

M: Slowscan length of the blur filter

X: Current column position

Y: Current scanline position

FSBLUR_(X,Y): Value at the current column/scanline position of thefastscan blurred input video

FSBLUR_(X,(Y−(M−1)):) Value M−1 scanlines up from the current positionof the fastscan blurred input video.

SSPS_(X,Y:) Slowscan Lead Edge of context Partial Sum consisting of thesum of the M−1 fastscan blurred values in a column above the currentposition.

The slowscan blur filter output consisting of the sum of the M Fastscanblur filter output values trailing and including the pixel at positionX,Y is SSBLUR_(X,Y)=SSPS_(X,Y)+FSBLUR_(X,Y), and the new partial sum forthe next row, same column position isSSPS_(X,(Y+1))=SSBLUR_(X,Y)−FSBLUR_(X,(Y−(M−1))).

In the fastscan case, the new partial sum would be used on the nextpixel processed. However in the slowscan case, because of thefastscan/slowscan raster presentation of the data, the column-wisepartial sum must be saved until the same column in the next row isprocessed. Since each value is created only once, used only once, andevery value is used P−1 pixels after it is created, the column partialsums can be saved in a FIFO memory. For each column, the partial sum forthe current position is stored during the previous row. This value isread from one end of the FIFO, and the new partial sum calculated forthe current column is written to the other. The next FIFO read will bethe partial sum for the next column position. These operations aresaving and restoring the state of the P slowscan blur filters whichexist for each of the P pixel columns in the image.

It is noted that while the overall two-dimensional blur filter requiresstorage for M−1 scanlines of the original image, access is only neededat the upper and lower right corners of the blur rectangle. A FIFOmemory can be used, thereby reducing control circuitry and Input/Outputpins required to a minimum: data input/output and simple read/writesignals. If multiple passes through a rectangular blur filter arerequired to achieve a frequency response with lower side lobes, a secondinstance of the process described above can be implemented concurrently,with the M scanline FIFO memory element and the per column accumulatorstorage being made wider but using the same control circuitry as thefirst stage. In this case, the present invention's use of a duplicatefastscan blur block in the second stage saves ┌Log₂ M*N┐ bits in widthof the M scanlines of storage. If N equals 10 and M equals 15, the Mscanline FIFO memory element width required by the second filter isreduced by ┌Log₂150┐=8 bits over the conventional sequence approach ofimplementing a separable two-dimensional filter.

Advantages from the above-described 2-D blur filter implementation arethat access to the data is required only after a single fix delay,thereby allowing the use of FIFO storage elements. Replication ofcritical functions reduces the required width of the FIFO storageelements. Regularity in the process utilizing the rectangular blurfilter allows multiple instances of the filter to be ganged together,thereby sharing control circuitry in processing the data concurrently torealize multiple passes through the filter without a correspondingincrease in cost and complexity.

Moreover, peak detection routines are often used in image processing tohelp find maximum and minimums of the images. In halftone detectionalgorithms, the peak detection is often used in determining the dotdensity or frequency of the halftone being measured. This is a commonmeasurement used in segmentation. In many detection routines, themaximum is found by searching for a local pixel that is greater than allits neighbors.

The problem with this process is that it ignores flat top peaks; i.e.,peaks that are more than 1 pixel in width. In order to overcome thisproblem, one can require that the pixel is greater than or equal to allof its neighbors. The problem associated with this implementation isthat the flat top area may be present as multiple pixels. As an example,utilizing a peak function P(i,j) which is one where there is a maximum,and zero otherwise, a typical peak detection process for pixel in thei-th scanline j-th pixel location is as follows:

Peak(i,j)=1 iff v(i,j)>v(i−k,j−1), with −1≦k, 1≦1 and k and | are notboth zero, otherwise 0

If there are dual peaks, for example: v(i,j)=v(i+1,j), neither (i,j) or(i+1,j) will be considered locations of a local maximum. This processwill therefore tend to underestimate the number of local maximums of theimage. In halftone measurements, this would result in a frequencyestimate lower than the true frequency. Similarly, if the strictinequality in the equation set forth above is changed to a weakinequality (≧), all flat top peaks will be counted multiple times, andthis will tend to overestimate the number of local maximums in theimage.

To correct for these problems adjacent peaks can be disallowed where thepeak pixel and all intervening pixels are at the same video level asanother peak. This unfortunately increases the complexity of the processand requires that the local peak mass be stored for future calculations.In terms of implementation, this is costly because storing line buffersfor real-time calculations is expensive in high speed systems.

One embodiment of the present invention resolves this problem with acombination of the strict and weak inequality. The implementation of theweak and equality can be direction sensitive. It is only imposed at aparticular direction: e.g., down and/or to the right. In this case, theinequality in the above equation is changed to:

Peak(i,j)=1 iff

v(i,j)≧v(i+k,j+1), with (k,1)={(0,1), (1,−1), (1,0), (1,1)}

v(i,j)>v(i+k,j+1), with (k,1)={(−1,−1), (−1,0), (−1,1), (0,−1)}

This inequality will count multiple peaks only one time, since it onlyallows it as it approaches from only one direction. In this example, itwill consider the upper left to be the peak location. Unfortunately,this process has also a shortcoming. The process will detect peaks inareas where staggered lines are encountered. The upper left area of eachjagged section will be detected as a local maximum. This type ofstaggering is, for example, in text areas where diagonal lines arepresent such as in part of the letter A. In many instances, thedetection of these areas as peaks many not be undesirable. For halftonedetection, however, it may result in a false detection of a halftonearea.

In order to eliminate detecting local maximums in this area, the secondequation above is updated to place the diagonal inequality in the upperleft. This eliminates block areas that are typically present in text.The updated peak detection is as follows:

Peak(i,j)=1 iff

v(i,j)>=v(i+k,j+1), with (k,1)={(−1,−1), (−1,1), (0,1), (1,0)}

v(i,j)>v(i+k,j+1), with (k,1)={(−1,0),(0,−1), (1,−1), (1,1)}

If this peak detection is utilized on a halftone gray original at 100lines per inch with 45 degree orientation. Under ideal peak detection,the peak map will form a hexagonal grid. Thus, given perfect peakdetection, the frequency measurement would have a mean of 100 and astandard deviation of 0. Using the first described peak detectionprocess above, the mean of the frequency measurement would be 91.5 linesper inch, the range 61 lines per inch to 100 and 9 lines per inch andthe standard deviation 5.6 lines per inch. Using the third describedpeak detection, the mean frequency is 99 lines per inch, range 91 linesper inch to 111 lines per inch and the standard deviation is 2 lines perinch.

As noted above, the detection of halftones is very important. Thedetection of halftones involves finding the center of the halftone dotsand counting the number of dot centers in a small window. The presenceof enough numbers of dots as well as additional conditions qualifies anarea as a halftone. The frequency of the halftone can be inferred fromthe dot counts in the window of a known size. The dot centers areidentified by finding the local video minimums in the mid-tone tohighlight area and finding the local video maximums in the mid-tone toshadow areas. To suppress false detections in a noisy non-halftone area,the center pixel is required to be less than the average of thesurrounding pixels by a small threshold in case of minimums and greaterthan the average of the surrounding pixels by a threshold in case ofmaximums.

The process described above usually detects the halftone areaaccurately. However, in images with small kanji, thin lines, or laddercharts, noises in the lines can trigger false detection of halftone dotcenters. If there are enough halftone peaks, the local area could beclassified as halftone rather than text or line arts. The kanji orladder charts would then be treated as high frequency halftones, therebycausing the kanji or ladder chart image data to be low pass filtered andscreened. The result is the loss of sharpness in reproduction. Thus,additional constraints are necessary in qualifying the pixel as ahalftone peak. One embodiment of the present invention provides anefficient way to reduce the number of false detections of halftones inthese areas.

To explain the present invention more clearly, the following examplewill be discussed. As shown in FIG. 13, FIG. 13 illustrates a halftoneimage which is reproduced at 133 lines per inch and has been magnified.Using the peak detection process described above, the pixels detected ashalftone peaks are shown as black dots in FIG. 14. The result ofapplying the simple halftone peak detection to a ladder chart as shownin FIG. 15, and kanji shown in FIG. 18. The false detection of halftonepeaks are obvious from the illustrations of FIGS. 16 and 19 whereinthese Figures represent maps of the detected halftone peaks. To suppressthese false detections, it is necessary to apply additional criteria toqualify a pixel as a halftone peak.

If the video signal is denoted by V_(i,j) wherein i is the _(I) scanline and j is the j pixel, the false detections can be suppressed bycomputing the range of the video in the horizontal, vertical, anddiagonal directions across three or five adjacent pixels. For example:for 400 spi R_(Horizontal)=Range(V_(i,j−1), V_(i,j), V_(i,j+1)) andR_(Vertical)=Range(V_(i−1,j), V_(i,j), V_(i+1,j)); for 600 spiR_(Horizontal)=Range(V_(i,j−2), V_(i,j−1), V_(i,j), V_(i,j+1),V_(i,j+2)) and R_(Vertical)=Range(V_(i−2,j), V_(i−1,j), V_(i,j),V_(i+1,j), V_(i+2,j)); R_(Diagonal1)=Range(V_(i−1,j+1), V_(i,j),V_(i+1,j−1)); and R_(Diagonal2)=Range(V_(i+1,j−1), V_(i,j),V_(i−1,j+1)).

The variations of the video across a few pixels along the direction of aline is expected to be small. If any of these computed video ranges issmall at a pixel location, it is likely that a line has passed throughthe pixel location. These ranges are all expected to be somewhat largerwhen the center pixel is a real halftone peak. Therefore, the followingadditional condition is imposed upon the center pixel to be classifiedas a halftone peak.

R_(Horizontal)>S and R_(Vertical)>S and R_(Diagonal1)>S andR_(Diagonal2)>S

With this additional constraint, the halftone peak detections for theimages shown in FIGS. 15 and 18 are shown in FIGS. 17 and 20,respectively. The false halftone peak detections are significantlyreduced. On the other hand, for a real halftone image as illustrated inFIG. 13, the additional constraints put on the center pixel do notreduce the number of halftone peaks detected.

As described above, a compound document can be automatically segmentedinto areas of different classes. Several of the classifiers(microclassifiers) used in this process depend on the video ofneighboring pixels. Since these classifiers (microclassifiers) depend onthe video of neighboring pixels, these classifiers (microclassifiers)are very sensitive to the resolution of the actual scanned in document.For example, the common scanning resolutions are 400 spots per inch and600 spots per inch. If a classifier (microclassifier) is built tocorrespond to a 400 spot per inch resolution, the classifier(microclassifier) may not provide a correct or optimal output if theimage being fed through the classifier is 600 spots per inch.

Therefore, it is desirable that the classifiers (microclassifiers) areimplemented such that the classifiers (microclassifiers) are made lesssensitive to the actual resolution of the video signal being processed.One embodiment of the present invention realizes this desirability byadjusting the size of the context or number of neighboring pixels whichwill be utilized by the classifier (microclassifier) in proportion tothe scanning resolution of the image. By adjusting the size of thecontext in proportion to the scanning resolution of the image, thecomputed classifier (microclassifier) can be made less sensitive to theactual resolution. More specifically, the classifier (microclassifier)of the present invention has the built in flexibility such that thecombination of these resolutions in either fastscan or slowscandirections can be readily accommodated by merely adjusting the size ofthe context in which the classifier (microclassifier) operates upon.

For example, in a halftone peak detection circuit, the halftone area isdetected by the presence of enough halftone dots in a window of knownsize. The halftone dots are identified by the local minimum of the videoin the mid-tone to highlight area and the local maximum of the video inthe mid-tone to shadow area. If the video signal is denoted by V_(i,j)for the j-th pixel in the i-th scanline, one of the necessary conditionsfor the local video minimum is as follows:

V_(i−1,j−1)≦V_(i,j) and V_(i−1,j)<V_(i,j) and V_(i−1,j+1)≦V_(i,j) andV_(i,j−1)<V_(i,j) and V_(i,j+1)≦V_(i,j) and V_(i+1,j−1)<V_(i,j) andV_(i+1,j)≦V_(i,j) and V_(i+1,j+1)<V_(i,j)

For a resolution of 400 spots per inch, the checking specified above isadequate. However, if the scanning resolution is increased to 600 spotsper inch, the necessary conditions must be extended to include morepixels, namely V_(i,j−2)<V_(i,j) and V_(i,j+2)≦V_(i,j) for the fastscandirection and V_(i−2,j)<V_(i,j) and V_(i+2,j)≦V_(i,j) for the slowscandirection. For the local maximum of the video similar conditions must bealso extended wherein the less than and less than and equal are replacedby greater than and greater than or equal, respectively.

Furthermore, in order to reduce the false peak detection in areas ofkanji, thin lines and ladder charts, as discussed above, the followingconditions are also analyzed wherein S is a small threshold.

Range(V_(i,j−1), V_(i,j), V_(i,j+1))>S and Range(V_(i−1,j), V_(i,j),V_(i+1,j))>S

As noted before, if the image has a resolution of 400 spots per inch,the conditions set forth above are adequate. However, if the image has aresolution of 600 spots per inch, it is also necessary to include morepixels in this condition, namely Range(V_(i,j−2), V_(i,j−1), V_(i,j),V_(i,j+1), V_(i,j+2)) and Range(V_(i−2,j), V_(i−1,), V_(i,j), V_(i+1,j),V_(i+2,j)) for the higher resolution in the fastscan and slowscandirections, respectively.

As noted above, the number of local video minimums and maximums in awindow of known size can be used to estimate the halftone frequency. Inactual implementation, a double convolution using a window of uniformweights is applied to 1 bit per pixel bitmap of halftone peaks. This isequivalent to sampling the peak counts with a window of twice the sizewith a triangular weighting function. The size of the window for theconvolution can be 8, 10, 12, 15, or 18 pixels in a fastscan directionand 8, 10, 12, 15 or 18 scanlines in the slowscan direction. Typically,if the scanning resolution of the image is 400 spots per inch, thewindow size for this implementation is usually 10 pixels in the fastscandirection and 10 scanlines in the slowscan direction. On the other hand,if the image has a resolution of 600 spots per inch, the typical windowis 15 pixels in the fastscan direction and 15 scanlines in the slowscandirection. Thus, as noted above, to properly detect the halftone peakcounts, the context of the window being analyzed must be adjustedaccording to the resolution of the incoming image.

To characterize continuous tone verses halftone, text and images, ameasure of energy or variation of the video is computed. In thisimplementation, the Laplacian of the video is first computed as follows:

L_(i,j)=(V_(i−1,j−1)+V_(i−1,j)+V_(i−1,j+1)+V_(i,j−1)+V_(i,j+1)+V_(i+1,j−1)+V_(i+1,j)+V_(i+1,j+1))/8−V_(i,j)

More specifically, the Laplacian is simply the difference between apixel and the average of its eight neighbors. After computing theLaplacian, the sum of the absolute values of the Laplacians over a smallneighborhood of N_(X) pixels by N_(Y) scanlines surrounding the pixel ofinterest is computed. The parameters N_(X) is chosen to be 7 for a 400spot per inch image and 11 for a 600 spot per inch image. The parametersN_(Y) are chosen to be 3 for a 400 spot per inch image and 5 for a 600spot per inch image. A pixel is classified as either a smooth contone,rough contone, or edge class depending on its value of the absolute sum.Large absolute Laplacian sums are also a necessary condition for a pixelto be classified as a halftone.

Lastly, it is also desirable to detect line, text, or edges inside ahalftone image or tint. Such a detection can be realized by firstcalculating a local average of the video over a suitable chosen contextwherein the local average is computed for each pixel. Next the range ofthe computed averages over a small neighborhood is checked. If the rangeexceeds a certain threshold, it is likely that the pixel is an edgepixel. For high frequency halftone tint, the context for averaging thevideo is three pixels or scanlines for an image at 400 spots per inchrevolution and 5 pixels or scanlines for an image at 600 spots per inchresolution in order to effectively smooth out the halftone tint area.

As readily seen above, the various classifiers utilized in anauto-segmentation process requires the context of the window beinganalyzed to vary from resolution to resolution of the input image.Therefore, the present invention adjusts the context of the window basedon the resolution of the input image. In other words, the implementationof the automatic image segmentation process has the built in flexibilityto effectively analyze the image at any input resolution wherein this isaccomplished by varying the context required for the classifiers(microclassifiers) that the determine the segmentation in proportion tothe input resolution.

An example of such a system is illustrated in FIG. 12. As illustrated inFIG. 12, the video signal or image data is fed into a microclassifiersystem 100, a macro-reduction system 200, and a pixel classificationlook-up table 300. The microclassifier system 100 is made of a pluralityof microclassifier circuits 101-104. These microclassifier circuitsmeasure intrinsic properties of the video extracted through a simplemathematical formula or heuristic. The microclassification values arefed into the macro-reduction system 200. The microclassifier system 100also receives the resolution of the incoming video signal. Thisinformation is used by the microclassifier circuits to conform theircalculations and measurements to the actual resolution of the videosignal.

The macro-reduction system 200 is made of a plurality of macro-reductioncircuits 201-204. Each macro-reduction circuit reducesmicroclassification values received from the microclassifiers to producehigher level, more directly useful information through mathematicaloperations and heuristics, suppressing noise and other irrelevant orundesirable variations in the microclassifiers. The macro-reduced valuesare fed into the pixel classification look-up table 300. To the extentto which the expected microclassifier values are impacted by theresolution driven adjustment of their calculation, the macro-reductionparameters must be adjusted by either a control processor or adaptedaccordingly by the system.

The pixel classification look-up table 300 is a programmable look-uptable reduces the macro-reduced outputs to the final classification in avery fast manner. The look-up table 300 also enables any arbitrarilycomplex function to be performed, since the arbitrarily complex functioncan be pre-calculated off-line and loaded into the table.Microclassification values or even the input video may also be useddirectly by the programmable table 300 if appropriate.

Although the present invention has been described in detail above,various modifications can be implemented without departing from thespirit of the present invention. For example, the preferred embodimentof the present invention has been described with respect to axerographic printing system; however, these fuzzy methods and filtersare readily implemented in a thermal inkjet system, a display system, orother image processing system

Moreover, the image processing system of the present invention can bereadily implemented on a general purpose computer, a personal computeror workstation. The image processing system of the present invention canbe readily implemented on an ASIC, thereby enabling the placement ofthis process in a scanner, electronic subsystem, printer, or displaydevice.

The present invention has been described with respect to a video rangeof 0 to 255. However, it is contemplated by the present invention thatthe video range can be any suitable range to describe the grey level ofthe pixel being processed. Furthermore, the present invention is readilyapplicable to any image processing system, not necessarily a binaryoutput device. It is contemplated that the concepts of the presentinvention are readily applicable to a four-level output terminal orhigher.

Also, the present invention has been described, with respect to thefuzzy classification and fuzzy processing routines, that the scalarvalues are determined using the weighted sum of the centriod methodsince the centriods in the preferred embodiment are non-overlapping (theclasses are non-overlapping). However, the present invention is readilyapplicable to a system with overlapping classes. Such an extension isreadily known to those skilled in the art of fuzzy logic.

Lastly, the present invention has been described with respect to amonochrome or black/white environment. However, the concepts of thepresent invention are readily applicable to a color environment. Namely,the processing operations of the present invention can be applied toeach color space value, some function of a given pixel's color spacecomponents, or even a function of the color space components of a pixeland other pixels in the neighborhood.

While the invention has been described with reference to variousembodiments disclosed above, it is not confined to the details set forthabove, but is intended to cover such modifications or changes as maycome within the scope of the attached claims.

What is claimed is:
 1. A system for classifying a pixel of image data asone of a plurality of image types, comprising: a plurality ofmicroclassifiers, each microclassifier determining a distinct imagecharacteristic value of the pixel; a plurality of macroreductioncircuits operatively connected to said plurality of microclassifiers,each macroreduction circuit performing further higher level operationsupon the distinct image characteristic values of the pixel to producereduced values; and a classification circuit to classify the pixel as animage type based on the reduced values from said macroreductioncircuits.
 2. The system as claimed in claim 1, further comprising: aprogrammable logic gate array; said programmable logic gate arrayproviding programmable data pathways between said microclassifiers andsaid macroreduction circuits.
 3. The system as claimed in claim 1,wherein said classification circuit is a look-up table.
 4. The system asclaimed in claim 1, wherein said classification circuit is aprogrammable look-up table.
 5. The system as claimed in claim 1, whereinsaid plurality of microclassifiers includes: a first microclassifiercircuit to determine a video peak for a first window of pixels centeredon the pixel; a second microclassifier circuit to determine a videoaverage for a second window of pixels centered on the pixel; a thirdmicroclassifier circuit to determine a video minimum for a third windowof pixels centered on the pixel; and a fourth microclassifier circuit todetermine a Laplacian for a fourth window of pixels centered on thepixel.
 6. The system as claimed in claim 1, wherein said plurality ofmacroreduction circuits includes: a first macroreduction circuit todetermine a video peak count for the pixel; and a second macroreductioncircuit to determine an accumulative video average for the pixel.
 7. Asystem for classifying a pixel of image data as one of a plurality ofimage types, comprising: first means for determining a plurality ofdistinct image characteristic values of the pixel; second means,operatively connected to said first means, for performing further higherlevel operations upon the distinct image characteristic values of thepixel to produce reduced values; and a classification circuit toclassify the pixel as an image type based on the reduced values fromsaid macroreduction circuits.
 8. The system as claimed in claim 7,further comprising: a programmable logic gate array; said programmablelogic gate array providing programmable data pathways between said firstmeans and said second means.
 9. The system as claimed in claim 7,wherein said classification circuit is a look-up table.
 10. The systemas claimed in claim 7, wherein said classification circuit is aprogrammable look-up table.
 11. The system as claimed in claim 7,wherein said first means includes: a first microclassifier circuit todetermine a video peak for a first window of pixels centered on thepixel; a second microclassifier circuit to determine a video average fora second window of pixels centered on the pixel; a third microclassifiercircuit to determine a video minimum for a third window of pixelscentered on the pixel; and a fourth microclassifier circuit to determinea Laplacian for a fourth window of pixels centered on the pixel.
 12. Thesystem as claimed in claim 7, wherein said second means includes: afirst macroreduction circuit to determine a video peak count for thepixel; and a second macroreduction circuit to determine an accumulativevideo average for the pixel.
 13. A system for determining a membershipvalue for a classification vector associated with a pixel of image data,comprising: a plurality of microclassifiers, each microclassifierdetermining a distinct image characteristic value of the pixel; aplurality of macroreduction circuits operatively connected to saidplurality of microclassifiers, each macroreduction circuit performingfurther higher level operations upon the distinct image characteristicvalues of the pixel to produce reduced values; and a classificationcircuit to assign a membership value to the classification vectorassociated with the pixel based on the reduced values from saidmacroreduction circuits.
 14. The system as claimed in claim 13, furthercomprising: a programmable logic gate array; said programmable logicgate array providing programmable data pathways between saidmicroclassifiers and said macroreduction circuits.
 15. The system asclaimed in claim 13, wherein said classification circuit is a look-uptable.
 16. The system as claimed in claim 13, wherein saidclassification circuit is a programmable look-up table.
 17. The systemas claimed in claim 13, wherein said plurality of microclassifiersincludes: a first microclassifier circuit to determine a video peak fora first window of pixels centered on the pixel; a second microclassifiercircuit to determine a video average for a second window of pixelscentered on the pixel; a third microclassifier circuit to determine avideo minimum for a third window of pixels centered on the pixel; and afourth microclassifier circuit to determine a Laplacian for a fourthwindow of pixels centered on the pixel.
 18. The system as claimed inclaim 13, wherein said plurality of macroreduction circuits includes: afirst macroreduction circuit to determine a video peak count for thepixel; and a second macroreduction circuit to determine an accumulativevideo average for the pixel.
 19. A printing system for rendering a pixelof image data according to one of a plurality of image types,comprising: a plurality of microclassifiers, each microclassifierdetermining a distinct image characteristic value of the pixel; aplurality of macroreduction circuits operatively connected to saidplurality of microclassifiers, each macroreduction circuit performingfurther higher level operations upon the distinct image characteristicvalues of the pixel to produce reduced values; a classification circuitto classify the pixel as an image type based on the reduced values fromsaid macroreduction circuits; processing means for image processing thepixel according to the image type classification; and print means forrendering the processed pixel.